vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #491
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.ry8R0PegHL', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 7f6b664e-07cc-4ba2-b078-4bddaa18dd4e16gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618gitlab # 19gitlab # Allocating group tables: 0/8 done20gitlab # Writing inode tables: 0/8 done21gitlab # Creating journal (8192 blocks): done22gitlab # Writing superblocks and filesystem accounting information: 0/8 done23(finished: start all VMs, in 59.85 seconds)24gitlab # 25gitlab # Virtualisation disk image created.26gitlab: waiting for unit setup-gitlab.service27gitlab: waiting for the VM to finish booting28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kk6xi66bpq6cxspz549ja9s57p2s98k8-nixos-system-gitlab-test/init regInfo=/nix/store/qbc2gdsdxv38k2da24b3cn5jh3zdjmym-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000001] kvm-clock: using sched offset of 534646803 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor67gitlab # [ 0.000811] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000880] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002736] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002747] Using GB pages for direct mapping73gitlab # [ 0.002835] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002843] ACPI: Early table checksum verification disabled75gitlab # [ 0.002847] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002851] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002855] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002862] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002865] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002866] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002868] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002869] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002871] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002872] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002873] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002873] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002874] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002874] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003111] No NUMA configuration found90gitlab # [ 0.003112] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003115] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005451] Zone ranges:93gitlab # [ 0.005452] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005454] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005455] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005456] Device empty97gitlab # [ 0.005457] Movable zone start for each node98gitlab # [ 0.005458] Early memory node ranges99gitlab # [ 0.005458] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005459] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005460] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005461] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005481] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005754] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.117037] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.117482] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.117497] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.117531] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.117534] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.117536] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.117537] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.117538] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.117539] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.117541] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.117543] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.117547] TSC deadline timer available117gitlab # [ 0.117552] CPU topo: Max. logical packages: 1118gitlab # [ 0.117553] CPU topo: Max. logical dies: 1119gitlab # [ 0.117553] CPU topo: Max. dies per package: 1120gitlab # [ 0.117556] CPU topo: Max. threads per core: 1121gitlab # [ 0.117557] CPU topo: Num. cores per package: 4122gitlab # [ 0.117557] CPU topo: Num. threads per package: 4123gitlab # [ 0.117558] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.117584] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.117603] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.117606] kvm-guest: setup PV sched yield127gitlab # [ 0.117622] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.117623] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.117624] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.117626] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.117628] Booting paravirtualized kernel on KVM132gitlab # [ 0.117631] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.122094] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.124262] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.124312] kvm-guest: PV spinlocks enabled136gitlab # [ 0.124314] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.124317] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kk6xi66bpq6cxspz549ja9s57p2s98k8-nixos-system-gitlab-test/init regInfo=/nix/store/qbc2gdsdxv38k2da24b3cn5jh3zdjmym-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.124418] Unknown kernel command line parameters "regInfo=/nix/store/qbc2gdsdxv38k2da24b3cn5jh3zdjmym-closure-info/registration", will be passed to user space.139gitlab # [ 0.124431] random: crng init done140gitlab # [ 0.124432] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.132948] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.137088] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.137164] software IO TLB: area num 4.144gitlab # [ 0.208537] Fallback order for Node 0: 0145gitlab # [ 0.208545] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.208547] Policy zone: Normal147gitlab # [ 0.210950] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.221032] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.234060] allocated 12582912 bytes of page_ext150gitlab # [ 0.244556] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.244558] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.245417] Dynamic Preempt: lazy153gitlab # [ 0.245578] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.245579] rcu: RCU event tracing is enabled.155gitlab # [ 0.245580] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.245581] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.245581] Rude variant of Tasks RCU enabled.158gitlab # [ 0.245582] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.245582] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.245583] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.245603] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.245605] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.245606] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.249982] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.250303] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.250311] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.250421] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.254016] Console: colour VGA+ 80x25169gitlab # [ 0.254019] printk: legacy console [tty0] enabled170gitlab # [ 0.284877] printk: legacy console [ttyS0] enabled171gitlab # [ 0.396381] ACPI: Core revision 20250807172gitlab # [ 0.397289] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.398941] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.399993] x2apic enabled175gitlab # [ 0.400767] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.401727] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.403020] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.404550] kvm-guest: setup PV IPIs179gitlab # [ 0.406127] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.407197] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns181gitlab # [ 0.409038] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)182gitlab # [ 0.410122] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.411166] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.413035] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.414039] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.415036] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.416035] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.417850] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.419035] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.420035] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.421043] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.422036] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.423035] active return thunk: srso_alias_return_thunk194gitlab # [ 0.424768] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.426035] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.427035] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.428035] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.429035] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.430035] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.431034] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.432035] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.433035] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.434035] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.435035] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.436035] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.437035] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.466799] Freeing SMP alternatives memory: 44K208gitlab # [ 0.467671] pid_max: default: 32768 minimum: 301209gitlab # [ 0.468143] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.469141] landlock: Up and running.211gitlab # [ 0.470035] Yama: becoming mindful.212gitlab # [ 0.471255] LSM support for eBPF active213gitlab # [ 0.472240] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.473170] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.476236] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.477640] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.478043] ... version: 2218gitlab # [ 0.478814] ... bit width: 48219gitlab # [ 0.479037] ... generic counters: 6220gitlab # [ 0.479794] ... generic bitmap: 000000000000003f221gitlab # [ 0.480037] ... fixed-purpose counters: 0222gitlab # [ 0.480818] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.481037] ... value mask: 0000ffffffffffff224gitlab # [ 0.481992] ... max period: 00007fffffffffff225gitlab # [ 0.482794] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.483156] signal: max sigframe size: 3376227gitlab # [ 0.484000] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.484705] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.485218] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.490590] smp: Bringing up secondary CPUs ...231gitlab # [ 0.491441] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.492041] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.495215] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.497038] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)235gitlab # [ 0.498399] Memory: 6011296K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved)236gitlab # [ 0.501038] devtmpfs: initialized237gitlab # [ 0.502259] x86/mm: Memory block size: 128MB238gitlab # [ 0.505216] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.507121] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.508147] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.509328] PM: RTC time: 09:49:51, date: 2026-11-10242gitlab # [ 0.512914] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.515338] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.516103] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.518106] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.519051] audit: initializing netlink subsys (disabled)247gitlab # [ 0.520193] audit: type=2000 audit(1794304191.707:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.520324] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.522038] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.523037] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.524037] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.525037] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.526068] cpuidle: using governor menu254gitlab # [ 0.528263] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.529285] PCI: Using configuration type 1 for base access256gitlab # [ 0.530039] PCI: Using configuration type 1 for extended access257gitlab # [ 0.531233] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.533411] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.534942] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.535929] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.536946] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.542567] ACPI: Added _OSI(Module Device)263gitlab # [ 0.544044] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.546044] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.553140] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.560328] ACPI: Interpreter enabled267gitlab # [ 0.561059] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.561823] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.562071] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.563044] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.564034] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.569692] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.570045] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.572474] acpiphp: Slot [3] registered275gitlab # [ 0.573100] acpiphp: Slot [4] registered276gitlab # [ 0.573884] acpiphp: Slot [5] registered277gitlab # [ 0.574625] acpiphp: Slot [6] registered278gitlab # [ 0.575087] acpiphp: Slot [7] registered279gitlab # [ 0.575880] acpiphp: Slot [8] registered280gitlab # [ 0.576615] acpiphp: Slot [9] registered281gitlab # [ 0.578079] acpiphp: Slot [10] registered282gitlab # [ 0.578861] acpiphp: Slot [11] registered283gitlab # [ 0.579635] acpiphp: Slot [12] registered284gitlab # [ 0.580080] acpiphp: Slot [13] registered285gitlab # [ 0.580842] acpiphp: Slot [14] registered286gitlab # [ 0.582068] acpiphp: Slot [15] registered287gitlab # [ 0.582820] acpiphp: Slot [16] registered288gitlab # [ 0.583057] acpiphp: Slot [17] registered289gitlab # [ 0.583813] acpiphp: Slot [18] registered290gitlab # [ 0.585112] acpiphp: Slot [19] registered291gitlab # [ 0.585873] acpiphp: Slot [20] registered292gitlab # [ 0.586601] acpiphp: Slot [21] registered293gitlab # [ 0.587059] acpiphp: Slot [22] registered294gitlab # [ 0.587834] acpiphp: Slot [23] registered295gitlab # [ 0.589058] acpiphp: Slot [24] registered296gitlab # [ 0.589792] acpiphp: Slot [25] registered297gitlab # [ 0.590057] acpiphp: Slot [26] registered298gitlab # [ 0.590823] acpiphp: Slot [27] registered299gitlab # [ 0.592066] acpiphp: Slot [28] registered300gitlab # [ 0.592805] acpiphp: Slot [29] registered301gitlab # [ 0.593066] acpiphp: Slot [30] registered302gitlab # [ 0.593821] acpiphp: Slot [31] registered303gitlab # [ 0.595052] PCI host bridge to bus 0000:00304gitlab # [ 0.595788] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.596038] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.598038] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.599038] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.600038] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.602050] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.603435] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.604544] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.606501] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.610812] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.611066] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.613038] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.614038] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.615037] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.616228] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.619067] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.622155] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.623311] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.625049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.626267] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.647136] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.648053] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.649059] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.650377] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.653092] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.658073] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.659032] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.660089] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.672069] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.674614] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.679076] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.680016] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.680881] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.683311] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.688075] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.689119] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.691066] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.693500] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.702069] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.704062] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.707100] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.714805] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.724082] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.726059] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.728118] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.734302] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.742674] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.743045] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.744060] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.747462] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.751074] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.751985] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.753059] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.755043] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.757316] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.760077] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.761060] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.766125] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.775085] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.778060] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.781098] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.787296] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.797091] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.799059] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.801123] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.813931] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.816378] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.818339] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.821304] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.823193] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.827256] iommu: Default domain type: Translated375gitlab # [ 0.829041] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.831498] ACPI: bus type USB registered377gitlab # [ 0.832187] usbcore: registered new interface driver usbfs378gitlab # [ 0.833145] usbcore: registered new interface driver hub379gitlab # [ 0.835072] usbcore: registered new device driver usb380gitlab # [ 0.837320] NetLabel: Initializing381gitlab # [ 0.837966] NetLabel: domain hash size = 128382gitlab # [ 0.838654] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.839112] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.840048] PCI: Using ACPI for IRQ routing385gitlab # [ 0.842056] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.843034] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.843034] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.845041] vgaarb: loaded389gitlab # [ 0.845794] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.847037] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.851113] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.852756] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.853524] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.855009] pnp: PnP ACPI init395gitlab # [ 0.856170] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.864405] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.866062] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.867138] NET: Registered PF_INET protocol family399gitlab # [ 0.869289] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.888046] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.889639] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.891146] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.894917] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.896360] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.897643] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.899137] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.900387] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.901838] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.902844] NET: Registered PF_XDP protocol family410gitlab # [ 0.903714] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.904829] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.905880] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.907063] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.908260] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.909665] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.910719] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.913271] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.915986] PCI: CLS 0 bytes, default 64419gitlab # [ 0.916815] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.917352] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.917914] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.918041] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns423gitlab # [ 0.941221] Initialise system trusted keyrings424gitlab # [ 0.942360] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.953664] Key type asymmetric registered426gitlab # [ 0.954455] Asymmetric key parser 'x509' registered427gitlab # [ 0.955391] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.956994] io scheduler mq-deadline registered429gitlab # [ 0.959022] io scheduler kyber registered430gitlab # [ 0.964086] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.965482] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.967809] Linux agpgart interface v0.103433gitlab # [ 0.968604] ACPI: bus type drm_connector registered434gitlab # [ 0.971148] usbcore: registered new interface driver usbserial_generic435gitlab # [ 0.972304] usbserial: USB Serial support registered for generic436gitlab # [ 0.973364] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 0.973364] Please enable it if your BIOS has the CPPC option.438gitlab # [ 0.975781] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 0.977255] drop_monitor: Initializing network drop monitor service440gitlab # [ 0.978513] NET: Registered PF_INET6 protocol family441gitlab # [ 0.980097] Segment Routing with IPv6442gitlab # [ 0.980803] In-situ OAM (IOAM) with IPv6443gitlab # [ 0.983091] IPI shorthand broadcast: enabled444gitlab # [ 0.987404] sched_clock: Marking stable (836010851, 150723233)->(1014729979, -27995895)445gitlab # [ 0.989203] registered taskstats version 1446gitlab # [ 0.990303] Loading compiled-in X.509 certificates447gitlab # [ 1.000454] Demotion targets for Node 0: null448gitlab # [ 1.001799] Key type .fscrypt registered449gitlab # [ 1.002664] Key type fscrypt-provisioning registered450gitlab # [ 1.003903] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.005118] ima: Allocated hash algorithm: sha1452gitlab # [ 1.006202] ima: No architecture policies found453gitlab # [ 1.007560] PM: Magic number: 2:523:828454gitlab # [ 1.009915] RAS: Correctable Errors collector initialized.455gitlab # [ 1.016514] clk: Disabling unused clocks456gitlab # [ 1.017447] PM: genpd: Disabling unused power domains457gitlab # [ 1.206259] Freeing initrd memory: 29392K458gitlab # [ 1.210132] Freeing unused decrypted memory: 2028K459gitlab # [ 1.213016] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.214187] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.216463] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.218272] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.269633] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.270827] Run /init as init process465gitlab # [ 1.298684] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.337828] fuse: init (API version 7.45)467gitlab # [ 1.346192] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.353586] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.356983] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.393966] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.730790] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.771917] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.774132] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.776367] systemd[1]: Running in initrd.475gitlab # [ 1.778931] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.781901] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.891902] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.935105] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.946255] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.947526] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.949084] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.950317] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.951506] systemd[1]: Reached target Path Units.484gitlab # [ 1.952428] systemd[1]: Reached target Slice Units.485gitlab # [ 1.953404] systemd[1]: Reached target Swaps.486gitlab # [ 1.954264] systemd[1]: Reached target Timer Units.487gitlab # [ 1.955306] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.956587] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.957836] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.958872] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.959920] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.960921] systemd[1]: Reached target Socket Units.493gitlab # [ 1.962920] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.965036] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 1.967051] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 1.969899] systemd[1]: Starting Journal Service...497gitlab # [ 1.973812] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 1.976425] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 1.985964] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 1.995304] systemd-journald[90]: Collecting audit messages is disabled.501gitlab # [ 1.999662] systemd[1]: Finished Create List of Static Device Nodes.502gitlab # [ 2.005038] systemd[1]: modprobe@configfs.service: Deactivated successfully.503gitlab # [ 2.014956] systemd[1]: Finished Load Kernel Module configfs.504gitlab # [ 2.018699] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config505gitlab # [ 2.029070] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...506gitlab # [ 2.034710] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.507gitlab # [ 2.037344] netfs: FS-Cache loaded508gitlab # [ 2.043005] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev509gitlab # [ 2.044128] 9pnet: Installing 9P2000 support510gitlab # [ 2.047332] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.511gitlab # [ 2.051303] systemd[1]: Starting Create Static Device Nodes in /dev...512gitlab # [ 2.052779] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.513gitlab # [ 2.054429] systemd[1]: Finished Load Kernel Module 9pnet_virtio.514gitlab # [ 2.062143] systemd[1]: Finished Load Kernel Modules.515gitlab # [ 1.911789] systemd-modules-l[ 2.063364] systemd[1]: Started Journal Service.516gitlab # oad[92]: Using 4 probe threads517gitlab # [ 1.914206] systemd-modules-load[92]: Inserted module 'virtio_balloon'518gitlab # [ 1.915263] systemd-modules-load[92]: Inserted module 'virtio_gpu'519gitlab # [ 1.916239] systemd-modules-load[92]: Inserted module 'dm_mod'520gitlab # [ 1.917260] systemd[1]: Starting Apply Kernel Variables...521gitlab # [ 1.922697] systemd[1]: Finished Create Static Device Nodes in /dev.522gitlab # [ 1.925118] systemd[1]: Reached target Preparation for Local File Systems.523gitlab # [ 1.926106] systemd[1]: Reached target Local File Systems.524gitlab # [ 1.926846] systemd[1]: Starting Create System Files and Directories...525gitlab # [ 1.928632] systemd[1]: Starting Rule-based Manager for Device Events and Files...526gitlab # [ 1.930501] systemd[1]: Finished Apply Kernel Variables.527gitlab # [ 1.946761] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 1.952189] systemd-udevd[110]: Using default interface naming scheme 'v261'.529gitlab # [ 1.962672] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 1.999586] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.000494] systemd[1]: Reached target System Initialization.532gitlab # [ 2.001275] systemd[1]: Reached target Basic System.533gitlab # [ 2.229210] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12534gitlab # [ 2.242804] serio: i8042 KBD port at 0x60,0x64 irq 1535gitlab # [ 2.243559] serio: i8042 AUX port at 0x60,0x64 irq 12536gitlab # [ 2.251092] virtio_blk virtio4: 4/0/0 default/read/poll queues537gitlab # [ 2.254871] uhci_hcd 0000:00:01.2: UHCI Host Controller538gitlab # [ 2.257119] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1539gitlab # [ 2.257215] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)540gitlab # [ 2.257474] uhci_hcd 0000:00:01.2: detected 2 ports541gitlab # [ 2.262889] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140542gitlab # [ 2.264364] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18543gitlab # [ 2.265782] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1544gitlab # [ 2.266969] usb usb1: Product: UHCI Host Controller545gitlab # [ 2.267673] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd546gitlab # [ 2.268581] usb usb1: SerialNumber: 0000:00:01.2547gitlab # [ 2.276996] SCSI subsystem initialized548gitlab # [ 2.277932] virtio_blk virtio5: 4/0/0 default/read/poll queues549gitlab # [ 2.292097] hub 1-0:1.0: USB hub found550gitlab # [ 2.293211] hub 1-0:1.0: 2 ports detected551gitlab # [ 2.299839] virtio_blk virtio5: [vdb] 11953360 512-byte logical blocks (6.12 GB/5.70 GiB)552gitlab # [ 2.149433] systemd[1]: Starting Virtual Console Setup...553gitlab # [ 2.309426] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0554gitlab # [ 2.322232] scsi host0: ata_piix555gitlab # [ 2.172914] (udev-worker)[129]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.556gitlab # [ 2.177140] (udev-worker)[129]: Network interface NamePolicy= disabled on kernel command line.557gitlab # [ 2.178536] systemd-vconsole-setup[143]: Configuration of first virt[ 2.329792] scsi host1: ata_piix558gitlab # ual console was skipped, ignoring remaining ones.[ 2.330694] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0559gitlab # 560gitlab # [ 2.181875] s[ 2.332352] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0561gitlab # ystemd[1]: Finished Virtual Console Setup.562gitlab # [ 2.184846] (udev-worker)[131]: Network interface NamePolicy= disabled on kernel command line.563gitlab # [ 2.191280] systemd[1]: Found device /dev/disk/by-label/nix-store.564gitlab # [ 2.193868] systemd[1]: Found device /dev/disk/by-label/nixos.565gitlab # [ 2.194732] systemd[1]: Reached target Initrd Root Device.566gitlab # [ 2.195543] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...567gitlab # [ 2.222704] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks568gitlab # [ 2.229327] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.569gitlab # [ 2.497080] ata2: found unknown device (class 0)570gitlab # [ 2.499544] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100571gitlab # [ 2.503677] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5572gitlab # [ 2.513872] usb 1-1: new full-speed USB device number 2 using uhci_hcd573gitlab # [ 2.568540] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray574gitlab # [ 2.595597] cdrom: Uniform CD-ROM driver Revision: 3.20575gitlab # [ 2.682979] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.686122] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.688824] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.690470] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.692043] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.726421] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.740070] usbcore: registered new interface driver usbhid582gitlab # [ 2.741030] usbhid: USB HID core driver583gitlab # [ 2.747276] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.751914] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.847137] systemd[1]: Mounting /sysroot...586gitlab # [ 3.154997] EXT4-fs (vda): mounted filesystem 7f6b664e-07cc-4ba2-b078-4bddaa18dd4e r/w with ordered data mode. Quota mode: none.587gitlab # [ 3.009856] systemd[1]: Mounted /sysroot.588gitlab # [ 3.011738] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 3.013887] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 3.015947] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.017953] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.019761] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.034669] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.041067] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.193750] erofs (device vdb): mounted with root inode @ nid 128.596gitlab # [ 3.047044] systemd[1]: Mounted /sysroot/nix/.ro-store.597gitlab # [ 3.048464] systemd[1]: Mounted /sysroot/nix/.rw-store.598gitlab # [ 3.049832] systemd[1]: Mounted /sysroot/run.599gitlab # [ 3.057276] systemd[1]: Starting rw-sysroot-nix-store.service...600gitlab # [ 3.059345] systemd-sysroot-fstab-check[177]: /sysroot should be mounted in the initrd, will request daemon-reload.601gitlab # [ 3.061691] systemd[1]: Reload requested from client PID 177 ('systemd-sysroot') (unit initrd-parse-etc.service)...602gitlab # [ 3.215171] 9p: Installing v9fs 9p2000 file system support603gitlab # [ 3.066174] systemd[1]: Reloading...604gitlab # [ 3.121373] systemd[1]: Reloading finished in 60 ms.605gitlab # [ 3.148548] systemd-sysroot-fstab-check[177]: Requesting initrd-fs.target/start/replace...606gitlab # [ 3.151373] systemd[1]: Mounted /sysroot/tmp/shared.607gitlab # [ 3.153183] systemd[1]: Mounted /sysroot/tmp/xchg.608gitlab # [ 3.154839] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.157361] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.159284] systemd-sysroot-fstab-check[177]: Requesting swap.target/start/replace...611gitlab # [ 3.161729] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.163787] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.166102] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.168263] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.175556] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.177904] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.847588] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.897439] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.899622] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.901750] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.903595] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.935552] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.938481] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.952053] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.953902] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.955948] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 3.993997] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 3.996213] systemd[1]: Stopped target Basic System.629gitlab # [ 3.998075] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 4.000135] systemd[1]: Stopped target Path Units.631gitlab # [ 4.001939] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 4.005477] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 4.008977] systemd[1]: Stopped target Slice Units.634gitlab # [ 4.012673] systemd[1]: Stopped target Socket Units.635gitlab # [ 4.014825] systemd[1]: Stopped target System Initialization.636gitlab # [ 4.017360] systemd[1]: Stopped target Swaps.637gitlab # [ 4.019113] systemd[1]: Stopped target Timer Units.638gitlab # [ 4.020935] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 4.023124] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 4.025430] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 4.028065] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.029857] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.033569] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.035536] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.037352] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.038837] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.040816] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.042328] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.044531] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.046832] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.048844] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.050591] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.052170] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.053903] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.054926] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.055742] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.056800] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.057832] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.058600] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.059606] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.060577] systemd[1]: systemd-udevd.service: Deactivated successfully.662gitlab # [ 4.061531] systemd[1]: Stopped Rule-based Manager for Device Events and Files.663gitlab # [ 4.062564] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.664gitlab # [ 4.063552] systemd[1]: Finished rw-sysroot-nix-store.service.665gitlab # [ 4.064398] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.065376] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.066107] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.066862] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.067922] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.068795] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.069950] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.070964] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.071921] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.072822] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.073880] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.074625] systemd[1]: Reached target Switch Root.677gitlab # [ 4.075437] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.195926] initrd-nixos-activation-start[227]: booting system configuration /nix/store/kk6xi66bpq6cxspz549ja9s57p2s98k8-nixos-system-gitlab-test679gitlab # [ 4.242048] initrd-nixos-activation-start[227]: running activation script...680gitlab # [ 4.521341] initrd-nixos-activation-start[250]: setting up /etc...681gitlab # [ 4.607805] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.610681] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.612556] systemd[1]: Starting Switch Root...684gitlab # [ 4.637751] systemd[1]: Switching root.685gitlab # [ 4.845324] systemd-journald[90]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.960571] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.334884] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.345505] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.347444] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.349521] systemd[1]: Detected first boot.691gitlab # [ 5.352205] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.513674] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.612664] systemd[1]: Applying preset policy.694gitlab # [ 5.752136] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.872257] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.873627] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.875864] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.879233] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.881171] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.882775] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.883753] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.885217] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.886468] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.887289] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.888089] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.888970] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.889816] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.890748] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.891647] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.892648] systemd[1]: Reached target Path Units.711gitlab # [ 5.893531] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.894452] systemd[1]: Reached target Slice Units.713gitlab # [ 5.895248] systemd[1]: Reached target Swaps.714gitlab # [ 5.896902] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.899088] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.900599] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.902379] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.903407] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.906076] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.907740] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.909813] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.911065] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.912302] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.913835] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.916603] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.917747] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.918688] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.921967] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.924263] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.926444] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.929988] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.937480] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.938655] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.942251] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.944067] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.945754] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.947746] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.951793] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.964863] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 5.971932] systemd[1]: Starting Journal Service...741gitlab # [ 5.977961] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 5.990644] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 5.996578] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 5.998202] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.006831] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.012231] loop: module loaded747gitlab # [ 6.013789] systemd[1]: Listening on Journal Log Access Socket.748gitlab # [ 6.014850] systemd[1]: Mounted Huge Pages File System.749gitlab # [ 6.015893] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 6.015952] systemd-journald[320]: Collecting audit messages is enabled.751gitlab # [ 6.016377] systemd[1]: Mounted Kernel Debug File System.752gitlab # [ 6.019106] systemd[1]: Mounted Kernel Trace File System.753gitlab # [ 6.020697] systemd[1]: Finished Create List of Static Device Nodes.754gitlab # [ 6.022040] systemd[1]: modprobe@configfs.service: Deactivated successfully.755gitlab # [ 6.023644] systemd[1]: Finished Load Kernel Module configfs.756gitlab # [ 6.025051] systemd[1]: Mounted FUSE Control File System.757gitlab # [ 6.026480] systemd[1]: Finished Load Kernel Modules.758gitlab # [ 6.030806] systemd[1]: Mounting Kernel Configuration File System...759gitlab # [ 6.034853] EXT4-fs (vda): re-mounted 7f6b664e-07cc-4ba2-b078-4bddaa18dd4e.760gitlab # [ 5.884834] systemd[1]: Queued start job for default target Multi-User System.761gitlab # [ 6.035875] systemd[1]: Starting Firewall...762gitlab # [ 5.887307] systemd[1]: Unnecessary job was removed for /dev/vdb.763gitlab # [ 5.888596] systemd[1]: systemd-journald.service: Deactivated successfully.764gitlab # [ 5.889927] systemd-modules-load[321]: Using 3 probe threads765gitlab # [ 5.890977] systemd-modules-load[321]: Inserted module 'tls'766gitlab # [ 5.892062] systemd-modules-load[321]: Inserted module 'loop'767gitlab # [ 5.893380] systemd-oomd[323]: No swap; memory pressure usage will be degraded768gitlab # [ 6.048620] systemd[1]: Starting Apply Kernel Variables...769gitlab # [ 6.052425] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...770gitlab # [ 6.056072] systemd[1]: Started Journal Service.771gitlab # [ 5.906730] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.772gitlab # [ 5.908918] systemd[1]: Finished Remount Root and Kernel File Systems.773gitlab # [ 5.910138] systemd[1]: Mounted Kernel Configuration File System.774gitlab # [ 5.919511] systemd[1]: Listening on Disk Image Download Service Socket.775gitlab # [ 5.922113] systemd[1]: Starting Flush Journal to Persistent Storage...776gitlab # [ 5.923713] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore777gitlab # [ 5.925625] systemd[1]: Starting Load/Save OS Random Seed...778gitlab # [ 5.926611] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os779gitlab # [ 5.928362] systemd[1]: Finished Apply Kernel Variables.780gitlab # [ 5.933063] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.781gitlab # [ 5.936286] systemd[1]: Starting Create Static Device Nodes in /dev...782gitlab # [ 6.092491] systemd-journald[320]: Received client request to flush runtime journal.783gitlab # [ 5.989203] systemd[1]: Finished Load/Save OS Random Seed.784gitlab # [ 5.991134] systemd[1]: Finished Create Static Device Nodes in /dev.785gitlab # [ 5.994252] systemd[1]: Reached target First Boot Complete.786gitlab # [ 5.995454] systemd[1]: Reached target Preparation for Local File Systems.787gitlab # [ 5.996542] systemd[1]: Starting Rule-based Manager for Device Events and Files...788gitlab # [ 5.997910] systemd-udevd[355]: Using default interface naming scheme 'v261'.789gitlab # [ 5.999158] systemd[1]: Finished Flush Journal to Persistent Storage.790gitlab # [ 6.012537] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.023639] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.088767] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.120288] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.169532] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.795gitlab # [ 6.183389] (udev-worker)[426]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 6.189832] (udev-worker)[421]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 6.192132] (udev-worker)[421]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.343309] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3799gitlab # [ 6.350929] ACPI: button: Power Button [PWRF]800gitlab # [ 6.409681] rtc_cmos 00:05: RTC can wake from S4801gitlab # [ 6.415284] Floppy drive(s): fd0 is 2.88M AMI BIOS802gitlab # [ 6.424816] rtc_cmos 00:05: registered as rtc0803gitlab # [ 6.425622] rtc_cmos 00:05: setting system clock to 2026-11-10T09:49:56 UTC (1794304196)804gitlab # [ 6.427240] FDC 0 is a S82078B805gitlab # [ 6.428819] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs806gitlab # [ 6.429388] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console807gitlab # [ 6.429951] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0808gitlab # [ 6.430032] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD809gitlab # [ 6.445880] Console: switching to colour dummy device 80x25810gitlab # [ 6.446665] [drm] Found bochs VGA, ID 0xb0c5.811gitlab # [ 6.447247] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.812gitlab # [ 6.448666] parport_pc 00:03: reported by Plug and Play ACPI813gitlab # [ 6.450767] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]814gitlab # [ 6.452167] mousedev: PS/2 mouse device common for all mice815gitlab # [ 6.452212] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic816gitlab # [ 6.453982] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0817gitlab # [ 6.454612] systemd-journald[320]: Time jumped backwards, rotating.818gitlab # [ 6.473199] Console: switching to colour frame buffer device 160x50819gitlab # [ 6.475030] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device820gitlab # [ 6.480925] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4821gitlab # [ 6.485271] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6822gitlab # [ 6.341053] systemd[1]: Condition check resulted in Virtio network device being skipped.823gitlab # [ 6.347824] systemd[1]: Finished Firewall.824gitlab # [ 6.349922] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoa[ 6.502044] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5825gitlab # ded=!efi_pstore826gitlab # [ 6.355252] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67827gitlab # [ 6.359027] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore828gitlab # [ 6.362453] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os829gitlab # [ 6.365293] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os830gitlab # [ 6.402776] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.590224] ppdev: user-space parallel port driver832gitlab # [ 6.459761] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.462651] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.469622] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 6.632938] kvm_amd: TSC scaling supported836gitlab # [ 6.633443] kvm_amd: Nested Virtualization enabled837gitlab # [ 6.634074] kvm_amd: Nested Paging enabled838gitlab # [ 6.634499] kvm_amd: LBR virtualization supported839gitlab # [ 6.635088] kvm_amd: Virtual VMLOAD VMSAVE supported840gitlab # [ 6.635587] kvm_amd: Virtual GIF supported841gitlab # [ 6.636096] kvm_amd: Virtual NMI enabled842gitlab # [ 6.673924] EDAC MC: Ver: 3.0.0843gitlab # [ 6.561165] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones.844gitlab # [ 6.563682] systemd[1]: Finished Virtual Console Setup.845gitlab # [ 6.726597] systemd[1]: Mounting /run/wrappers...846gitlab # [ 6.754713] systemd[1]: Mounted /run/wrappers.847gitlab # [ 6.756841] systemd[1]: Reached target Local File Systems.848gitlab # [ 6.758816] systemd[1]: Listening on Boot Loader Control Service Socket.849gitlab # [ 6.761262] systemd[1]: Starting register-nix-paths.service...850gitlab # [ 6.763655] systemd[1]: Starting Create SUID/SGID Wrappers...851gitlab # [ 6.765955] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852gitlab # [ 6.769377] systemd[1]: Starting Save Transient machine-id to Disk...853gitlab # [ 6.771881] systemd[1]: Starting Create System Files and Directories...854gitlab # [ 6.809470] systemd[1]: Finished Create System Files and Directories.855gitlab # [ 6.812528] systemd[1]: Starting Rebuild Journal Catalog...856gitlab # [ 6.815961] systemd[1]: Starting Record System Boot/Shutdown in UTMP...857gitlab # [ 6.818379] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 6.823857] systemd[1]: Finished Save Transient machine-id to Disk.859gitlab # [ 6.840830] systemd[1]: Finished Record System Boot/Shutdown in UTMP.860gitlab # [ 6.857191] systemd[1]: Finished Rebuild Journal Catalog.861gitlab # [ 6.859147] systemd[1]: Starting Update is Completed...862gitlab # [ 6.873162] systemd[1]: Finished Update is Completed.863gitlab # [ 6.925467] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.864gitlab # [ 6.927311] systemd[1]: Finished Create SUID/SGID Wrappers.865gitlab # [ 7.206113] systemd[1]: Finished register-nix-paths.service.866gitlab # [ 7.207236] systemd[1]: Reached target System Initialization.867gitlab # [ 7.208090] systemd[1]: Started Discard unused filesystem blocks once a week.868gitlab # [ 7.209232] systemd[1]: Started logrotate.timer.869gitlab # [ 7.209969] systemd[1]: Started Daily Cleanup of Temporary Directories.870gitlab # [ 7.211030] systemd[1]: Reached target Timer Units.871gitlab # [ 7.211795] systemd[1]: Listening on D-Bus System Message Bus Socket.872gitlab # [ 7.212872] systemd[1]: Listening on Nix Daemon Socket.873gitlab # [ 7.216029] systemd[1]: Starting nixbot CI service socket...874gitlab # [ 7.217044] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.875gitlab # [ 7.218209] systemd[1]: Listening on nixbot CI service socket.876gitlab # [ 7.219066] systemd[1]: Reached target Socket Units.877gitlab # [ 7.222532] systemd[1]: Starting D-Bus System Message Bus...878gitlab # [ 7.260177] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'...879gitlab # [ 7.261694] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync'880gitlab # [ 7.263295] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"881gitlab # [ 7.267916] systemd[1]: Started D-Bus System Message Bus.882gitlab # [ 7.280128] dbus-broker-launch[577]: Ready883gitlab # [ 7.283977] systemd[1]: Reached target Basic System.884gitlab # [ 7.287988] systemd[1]: Started backdoor.service.885gitlab # [ 7.293246] systemd[1]: Starting gitlab-config.service...886gitlab # [ 7.295196] systemd[1]: Starting Import lastlog data into lastlog2 database...887gitlab # [ 7.302165] systemd[1]: Starting Logrotate configuration check...888gitlab # [ 7.304634] systemd[1]: Starting Name Service Cache Daemon (nsncd)...889gitlab # [ 7.307283] systemd[1]: Starting Post-Boot Actions...890gitlab # connecting to host...891gitlab # [ 7.316931] systemd[1]: Started Reset console on configuration changes.892gitlab # [ 7.325265] systemd[1]: Starting resolvconf update...893gitlab # [ 7.333965] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.894gitlab # [ 7.335075] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf895gitlab # [ 7.337820] logrotate[582]: note: 'monthly' overrides previously specified 'weekly'896gitlab: Guest shell says: b'Spawning backdoor root shell...\n'897gitlab # [ 7.343201] logrotate[582]: Reading state from file: /var/lib/logrotate.status898gitlab # [ 7.343767] logrotate[582]: state file /var/lib/logrotate.status does not exist899gitlab: connected to guest root shell900gitlab # [ 7.344499] logrotate[582]: Allocating hash table for state file, size 64 entries901gitlab: (connecting took 8.04 seconds)902gitlab # [ 7.344990] logrotate[582]: Handling 4 logs903gitlab: (finished: waiting for the VM to finish booting, in 8.04 seconds)904gitlab # [ 7.345474] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed905gitlab # [ 7.345895] logrotate[582]: considering log /var/log/btmp906gitlab # [ 7.346599] logrotate[582]: Creating new state907gitlab # [ 7.346950] logrotate[582]: Now: 2026-11-10 09:49908gitlab # [ 7.347566] logrotate[582]: Last rotated at 2026-11-10 09:00909gitlab # [ 7.348029] logrotate[582]: log does not need rotating (log has already been rotated)910gitlab # [ 7.350576] systemd[1]: Finished Post-Boot Actions.911gitlab # [ 7.351597] systemd[1]: Finished Import lastlog data into lastlog2 database.912gitlab # [ 7.356717] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed913gitlab # [ 7.357626] logrotate[582]: considering log /var/log/wtmp914gitlab # [ 7.357898] logrotate[582]: Creating new state915gitlab # [ 7.358707] logrotate[582]: Now: 2026-11-10 09:49916gitlab # [ 7.360510] logrotate[582]: Last rotated at 2026-11-10 09:00917gitlab # [ 7.361107] logrotate[582]: log does not need rotating (log has already been rotated)918gitlab # [ 7.361797] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed919gitlab # [ 7.362456] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582)920gitlab # [ 7.366224] logrotate[582]: considering log /var/gitlab/state/log/*.log921gitlab # [ 7.366681] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping922gitlab # [ 7.367212] logrotate[582]: Creating new state923gitlab # [ 7.367737] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582)924gitlab # [ 7.368398] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed925gitlab # [ 7.369043] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582)926gitlab # [ 7.369514] logrotate[582]: considering log /var/log/nginx/*.log927gitlab # [ 7.370044] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping928gitlab # [ 7.370551] logrotate[582]: Creating new state929gitlab # [ 7.371057] logrotate[582]: not running postrotate script, since no logs were rotated930gitlab # [ 7.371574] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582)931gitlab # [ 7.372526] nsncd[583]: Nov 10 09:49:57.590 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"932gitlab # [ 7.373142] systemd[1]: Finished Logrotate configuration check.933gitlab # [ 7.373604] systemd[1]: Started Name Service Cache Daemon (nsncd).934gitlab # [ 7.374124] systemd[1]: Reached target Host and Network Name Lookups.935gitlab # [ 7.374645] systemd[1]: Reached target User and Group Name Lookups.936gitlab # [ 7.375183] systemd[1]: Starting User Login Management...937gitlab # [ 7.418410] systemd-logind[618]: New seat seat0.938gitlab # [ 7.421270] systemd-logind[618]: Watching system buttons on /dev/input/event2 (Power Button)939gitlab # [ 7.421819] systemd-logind[618]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)940gitlab # [ 7.422621] systemd-logind[618]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)941gitlab # [ 7.423088] systemd[1]: Started User Login Management.942gitlab # [ 7.423921] systemd[1]: Starting linger-users.service...943gitlab # [ 7.437961] systemd[1]: Stopped target Host and Network Name Lookups.944gitlab # [ 7.438613] systemd[1]: Stopping Host and Network Name Lookups...945gitlab # [ 7.439193] systemd[1]: Stopped target User and Group Name Lookups.946gitlab # [ 7.439668] systemd[1]: Stopping User and Group Name Lookups...947gitlab # [ 7.441790] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...948gitlab # [ 7.442546] systemd[1]: nscd.service: Deactivated successfully.949gitlab # [ 7.443516] systemd[1]: Stopped Name Service Cache Daemon (nsncd).950gitlab # [ 7.445211] systemd[1]: nscd.service: Consumed 30ms CPU time over 138ms wall clock time, 7.5M memory peak, 5.5M read from disk.951gitlab # [ 7.446087] systemd[1]: linger-users.service: Deactivated successfully.952gitlab # [ 7.446407] systemd[1]: Finished linger-users.service.953gitlab # [ 7.452947] systemd[1]: Starting Name Service Cache Daemon (nsncd)...954gitlab # [ 7.468738] nsncd[679]: Nov 10 09:49:57.693 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"955gitlab # [ 7.469697] systemd[1]: Started Name Service Cache Daemon (nsncd).956gitlab # [ 7.470217] systemd[1]: Reached target Host and Network Name Lookups.957gitlab # [ 7.470766] systemd[1]: Reached target User and Group Name Lookups.958gitlab # [ 7.476198] systemd[1]: Finished resolvconf update.959gitlab # [ 7.476924] systemd[1]: Reached target Preparation for Network.960gitlab # [ 7.480212] systemd[1]: Starting DHCP Client...961gitlab # [ 7.481597] systemd[1]: Starting Address configuration of eth1...962gitlab # [ 7.484190] systemd[1]: Starting Extra networking commands....963gitlab # [ 7.511958] network-addresses-eth1-start[701]: adding address 192.168.1.1/24... done964gitlab # [ 7.515679] network-addresses-eth1-start[701]: adding address 2001:db8:1::1/64... done965gitlab # [ 7.521230] systemd[1]: Finished Address configuration of eth1.966gitlab # [ 7.550203] dhcpcd[709]: dhcpcd-10.3.2 starting967gitlab # [ 7.553261] dhcpcd[758]: dev: loaded udev968gitlab # [ 7.709348] 8021q: 802.1Q VLAN Support v1.8969gitlab # [ 7.710001] 8021q: adding VLAN 0 to HW filter on device eth1970gitlab # [ 7.562825] systemd[1]: Finished Extra networking commands..971gitlab # [ 7.564932] systemd[1]: Reached target Network.972gitlab # [ 7.566704] systemd[1]: Starting gitlab-workhorse.service...973gitlab # [ 7.573066] systemd[1]: Starting Nginx Web Server...974gitlab # [ 7.576546] systemd[1]: Starting PostgreSQL Server...975gitlab # [ 7.580210] systemd[1]: Starting Redis Server - redis-gitlab...976gitlab # [ 7.583344] systemd[1]: Starting Permit User Sessions...977gitlab # [ 7.589536] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.978gitlab # [ 7.604563] systemd[1]: Finished Permit User Sessions.979gitlab # [ 7.623212] systemd[1]: Started Getty on tty1.980gitlab # [ 7.624197] systemd[1]: Reached target Login Prompts.981gitlab # [ 7.790440] cfg80211: Loading compiled-in X.509 certificates for regulatory database982gitlab # [ 7.796840] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'983gitlab # [ 7.799225] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'984gitlab # [ 7.800749] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2985gitlab # [ 7.801773] cfg80211: failed to load regulatory.db986gitlab # [ 7.830317] 8021q: adding VLAN 0 to HW filter on device eth0987gitlab # [ 7.683066] dhcpcd[758]: eth0: waiting for carrier988gitlab # [ 7.685309] dhcpcd[758]: eth0: carrier acquired989gitlab # [ 7.693946] dhcpcd[758]: DUID 00:01:00:01:32:85:a5:45:52:54:00:12:34:56990gitlab # [ 7.694817] dhcpcd[758]: eth0: IAID 00:12:34:56991gitlab # [ 7.695865] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456992gitlab # [ 7.710274] nginx-pre-start[795]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok993gitlab # [ 7.710984] nginx-pre-start[795]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful994gitlab # [ 7.717270] systemd[1]: Started Nginx Web Server.995gitlab # [ 7.732371] postgresql-pre-start[802]: The files belonging to this database system will be owned by user "postgres".996gitlab # [ 7.734139] postgresql-pre-start[802]: This user must also own the server process.997gitlab # [ 7.735247] redis-server[790]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.998gitlab # [ 7.735681] redis-server[790]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo999gitlab # [ 7.736324] redis-server[790]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=790, just started1000gitlab # [ 7.737111] redis-server[790]: Configuration loaded1001gitlab # [ 7.740740] postgresql-pre-start[802]: The database cluster will be initialized with locale "en_US.UTF-8".1002gitlab # [ 7.741826] postgresql-pre-start[802]: The default database encoding has accordingly been set to "UTF8".1003gitlab # [ 7.742530] postgresql-pre-start[802]: The default text search configuration will be set to "english".1004gitlab # [ 7.743162] postgresql-pre-start[802]: Data page checksums are enabled.1005gitlab # [ 7.744305] postgresql-pre-start[802]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1006gitlab # [ 7.747192] postgresql-pre-start[802]: creating subdirectories ... ok1007gitlab # [ 7.747917] postgresql-pre-start[802]: selecting dynamic shared memory implementation ... posix1008gitlab # [ 7.750323] redis[790]: monotonic clock: POSIX clock_gettime1009gitlab # [ 7.754860] redis[790]: Running mode=standalone, port=0.1010gitlab # [ 7.755761] redis[790]: Server initialized1011gitlab # [ 7.756493] systemd[1]: Started Redis Server - redis-gitlab.1012gitlab # [ 7.759185] redis[790]: Ready to accept connections unix1013gitlab # [ 7.761154] redis[790]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1014gitlab # [ 7.784421] postgresql-pre-start[802]: selecting default "max_connections" ... 1001015gitlab # [ 7.823633] postgresql-pre-start[802]: selecting default "shared_buffers" ... 128MB1016gitlab # [ 8.116592] systemd[1]: Started gitlab-workhorse.service.1017gitlab # [ 8.223741] postgresql-pre-start[802]: selecting default time zone ... UTC1018gitlab # [ 8.226287] postgresql-pre-start[802]: creating configuration files ... ok1019gitlab # [ 8.298354] gitlab-workhorse[825]: time="2026-11-10T09:49:58Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41020gitlab # [ 8.300216] gitlab-workhorse[825]: time="2026-11-10T09:49:58Z" level=info msg="Using redis/go-redis"1021gitlab # [ 8.302507] gitlab-workhorse[825]: time="2026-11-10T09:49:58Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1022gitlab # [ 8.414146] postgresql-pre-start[802]: running bootstrap script ... ok1023gitlab # [ 8.671433] dhcpcd[758]: eth0: soliciting a DHCP lease1024gitlab # [ 8.828255] NET: Registered PF_PACKET protocol family1025gitlab # [ 8.684747] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.21026gitlab # [ 8.690411] dhcpcd[758]: eth0: probing address 10.0.2.15/241027gitlab # [ 8.886879] postgresql-pre-start[802]: performing post-bootstrap initialization ... ok1028gitlab # [ 9.179584] postgresql-pre-start[802]: syncing data to disk ... ok1029gitlab # [ 9.180691] postgresql-pre-start[802]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 9.182063] postgresql-pre-start[802]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 9.184258] postgresql-pre-start[802]: Success. You can now start the database server using:1032gitlab # [ 9.185377] postgresql-pre-start[802]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 9.327437] postgres[853]: [853] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 9.329094] postgres[853]: [853] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 9.330191] postgres[853]: [853] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 9.336145] postgres[853]: [853] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 9.353948] postgres[862]: [862] LOG: database system was shut down at 2026-11-10 09:49:59 GMT1038gitlab # [ 9.365418] postgres[853]: [853] LOG: database system is ready to accept connections1039gitlab # [ 9.374260] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 9.382121] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 9.648138] postgresql-setup-start[882]: CREATE DATABASE1042gitlab # [ 9.677827] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[887]: mkdir -p /var/gitlab/state/home/.ssh: OK1043gitlab # [ 9.679423] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[887]: chmod 700 /var/gitlab/state/home/.ssh: OK1044gitlab # [ 9.691391] postgresql-setup-start[892]: CREATE ROLE1045gitlab # [ 9.700244] dhcpcd[758]: eth0: soliciting an IPv6 router1046gitlab # [ 9.701876] dhcpcd[758]: eth0: Router Advertisement from fe80::21047gitlab # [ 9.702831] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/641048gitlab # [ 9.704759] dhcpcd[758]: eth0: adding route to fec0::/641049gitlab # [ 9.706170] dhcpcd[758]: eth0: adding default route via fe80::21050gitlab # [ 9.718848] postgresql-setup-start[899]: ALTER DATABASE1051gitlab # [ 9.730078] systemd[1]: Finished gitlab-config.service.1052gitlab # [ 9.733673] systemd[1]: Started gitaly.service.1053gitlab # [ 9.746585] postgresql-setup-start[912]: CREATE ROLE1054gitlab # [ 9.750580] systemd[1]: Finished PostgreSQL Setup Scripts.1055gitlab # [ 9.752370] systemd[1]: Reached target PostgreSQL.1056gitlab # [ 9.754239] systemd[1]: Starting gitlab-db-config.service...1057gitlab # [ 9.759669] systemd[1]: Starting gitlab-postgresql.service...1058gitlab # [ 9.875815] gitlab-postgresql-start[923]: CREATE DATABASE1059gitlab # [ 9.909657] gitlab-postgresql-start[934]: CREATE EXTENSION1060gitlab # [ 10.005568] gitlab-postgresql-start[936]: CREATE EXTENSION1061gitlab # [ 10.009652] systemd[1]: Finished gitlab-postgresql.service.1062gitlab # [ 10.300289] gitaly[911]: time="2026-11-10T09:50:00.523Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9111063gitlab # [ 10.303397] gitaly[911]: time="2026-11-10T09:50:00.524Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9111064gitlab # [ 10.304727] gitaly[911]: time="2026-11-10T09:50:00.524Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9111065gitlab # [ 10.307245] gitaly[911]: time="2026-11-10T09:50:00.524Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9111066gitlab # [ 10.307982] gitaly[911]: time="2026-11-10T09:50:00.524Z" level=info msg="Starting Gitaly" pid=911 version=1067gitlab # [ 10.504349] gitaly[911]: time="2026-11-10T09:50:00.728Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=203 pid=9111068gitlab # [ 10.505223] gitaly[911]: time="2026-11-10T09:50:00.728Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9111069gitlab # [ 10.505874] gitaly[911]: time="2026-11-10T09:50:00.729Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9111070gitlab # [ 10.506500] gitaly[911]: time="2026-11-10T09:50:00.729Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-911/git-exec-2495290641.d/git pid=9111071gitlab # [ 10.513146] gitaly[911]: time="2026-11-10T09:50:00.737Z" level=info msg="finished detecting git version" duration_ms=8 pid=9111072gitlab # [ 10.514263] gitaly[911]: time="2026-11-10T09:50:00.737Z" level=info msg="using Git version" pid=911 version=2.53.g1073gitlab # [ 10.514869] gitaly[911]: time="2026-11-10T09:50:00.737Z" level=info msg="clearing disk cache object folder" pid=911 storage=default1074gitlab # [ 10.515709] gitaly[911]: time="2026-11-10T09:50:00.738Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3803160436 pid=911 storage=default1075gitlab # [ 10.516574] gitaly[911]: time="2026-11-10T09:50:00.738Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3803160436 pid=911 storage=default1076gitlab # [ 10.517310] gitaly[911]: time="2026-11-10T09:50:00.738Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9111077gitlab # [ 10.517719] gitaly[911]: time="2026-11-10T09:50:00.738Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9111078gitlab # [ 10.518439] gitaly[911]: time="2026-11-10T09:50:00.738Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3803160436 clear_duration_ms=0 pid=911 storage=default1079gitlab # [ 10.559730] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9111080gitlab # [ 10.560488] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=911 schema=unix1081gitlab # [ 10.560939] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="counting repositories" pid=9111082gitlab # [ 10.561550] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="starting to count repositories" pid=911 storage_path=/var/gitlab/state/repositories1083gitlab # [ 10.562062] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="finished tempdir cleaner walk" pid=911 storage=default time_ms=01084gitlab # [ 10.562517] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=911 storage_path=/var/gitlab/state/repositories1085gitlab # [ 10.562977] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="listening at address" address=/run/gitaly/gitaly-911/sock.d/intern pid=911 schema=unix1086gitlab # [ 10.563646] gitaly[911]: time="2026-11-10T09:50:00.784Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9111087gitlab # [ 10.564156] gitaly[911]: time="2026-11-10T09:50:00.786Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9111088gitlab # [ 10.564644] gitaly[911]: time="2026-11-10T09:50:00.786Z" level=info msg="Started Gitaly" duration_ms=261 pid=9111089gitlab # [ 10.565454] gitaly[911]: time="2026-11-10T09:50:00.786Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9111090gitlab # [ 10.565980] gitaly[911]: time="2026-11-10T09:50:00.786Z" level=info msg="maintenance: daily scheduled" pid=911 scheduled="2026-11-10 12:00:00 +0000 UTC"1091gitlab # [ 11.472459] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1092gitlab # [ 11.474462] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1093gitlab # [ 11.499701] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1094gitlab # [ 13.145913] gitaly[911]: time="2026-11-10T09:50:03.369Z" level=info msg="License database preloaded" duration_ms=2845 pid=9111095gitlab # [ 13.796575] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds1096gitlab # [ 13.798939] dhcpcd[758]: eth0: adding route to 10.0.2.0/241097gitlab # [ 13.800524] dhcpcd[758]: eth0: adding default route via 10.0.2.21098gitlab # [ 13.922701] systemd[1]: Started DHCP Client.1099gitlab # [ 13.924433] systemd[1]: Reached target Network is Online.1100gitlab # [ 43.837679] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: W, [2026-11-10T09:50:34.061809 #918] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1101gitlab # [ 46.343622] postgres[1339]: [1339] ERROR: relation "feature_gates" does not exist at character 5231102gitlab # [ 46.344173] postgres[1339]: [1339] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1103gitlab # [ 46.344452] postgres[1339]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1104gitlab # [ 46.346415] postgres[1339]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1105gitlab # [ 46.348583] postgres[1339]: attidentity AS identity,1106gitlab # [ 46.350371] postgres[1339]: attgenerated as attgenerated1107gitlab # [ 46.352378] postgres[1339]: FROM pg_attribute a1108gitlab # [ 46.354299] postgres[1339]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1109gitlab # [ 46.356301] postgres[1339]: LEFT JOIN pg_type t ON a.atttypid = t.oid1110gitlab # [ 46.357950] postgres[1339]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1111gitlab # [ 46.359990] postgres[1339]: WHERE a.attrelid = '"feature_gates"'::regclass1112gitlab # [ 46.361670] postgres[1339]: AND a.attnum > 0 AND NOT a.attisdropped1113gitlab # [ 46.364135] postgres[1339]: ORDER BY a.attnum1114gitlab # [ 46.366158] postgres[1339]:1115gitlab # [ 48.367850] gitlab-workhorse[825]: time="2026-11-10T09:50:38Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1116gitlab # [ 50.087600] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1381]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1117gitlab # [ 50.089811] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1381]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1118gitlab # [ 58.377697] gitlab-workhorse[825]: time="2026-11-10T09:50:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1119gitlab # [ 65.221731] postgres[1456]: [1456] ERROR: relation "table_without_models" does not exist1120gitlab # [ 65.223089] postgres[1456]: [1456] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1121gitlab # [ 68.387563] gitlab-workhorse[825]: time="2026-11-10T09:50:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 77.111310] systemd[1]: Finished gitlab-db-config.service.1123gitlab # [ 77.113680] systemd[1]: Started gitlab-sidekiq.service.1124gitlab # [ 77.115700] systemd[1]: Starting gitlab.service...1125gitlab # [ 77.489833] bundle[1623]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1126gitlab # [ 77.494104] bundle[1623]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1127gitlab # [ 77.498982] bundle[1623]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1128gitlab # [ 78.397815] gitlab-workhorse[825]: time="2026-11-10T09:51:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1129gitlab # [ 80.416954] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T09:51:10.616Z","message":"Starting cluster with 1 processes"}1130gitlab # [ 80.738554] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.935Z","pid":1623,"message":"* Enabling systemd notification integration"}1131gitlab # [ 80.739809] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"Puma starting in cluster mode..."}1132gitlab # [ 80.740495] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1133gitlab # [ 80.741219] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1134gitlab # [ 80.742064] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"* Min threads: 0"}1135gitlab # [ 80.743077] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"* Max threads: 4"}1136gitlab # [ 80.743903] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"* Environment: production"}1137gitlab # [ 80.744845] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.962Z","pid":1623,"message":"* Master PID: 1623"}1138gitlab # [ 80.745700] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.963Z","pid":1623,"message":"* Workers: 2"}1139gitlab # [ 80.746615] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.963Z","pid":1623,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1140gitlab # [ 80.747307] bundle[1623]: {"timestamp":"2026-11-10T09:51:10.963Z","pid":1623,"message":"* Preloading application"}1141gitlab # [ 88.399546] gitlab-workhorse[825]: time="2026-11-10T09:51:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1142gitlab # [ 98.409736] gitlab-workhorse[825]: time="2026-11-10T09:51:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143gitlab # [ 105.957878] bundle[1623]: W, [2026-11-10T09:51:36.181932 #1623] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1144gitlab # [ 107.457523] sidekiq-cluster[1653]: 2026-11-10T09:51:37.681Z pid=1653 tid=wkd INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1653, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>437362688}1145gitlab # [ 107.655200] sidekiq-cluster[1653]: W, [2026-11-10T09:51:37.879940 #1653] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1146gitlab # [ 107.702329] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:37.927Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1147gitlab # [ 108.419641] gitlab-workhorse[825]: time="2026-11-10T09:51:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1148gitlab # [ 109.912922] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.136Z","message":"GitLab reliable fetch activated!"}1149gitlab # [ 109.917474] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.137Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1150gitlab # [ 109.939859] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.164Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1151gitlab # [ 109.945260] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.168Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1152gitlab # [ 109.948736] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.173Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1153gitlab # [ 109.953813] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.178Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1154gitlab # [ 109.958886] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.183Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1155gitlab # [ 109.964061] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.189Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1156gitlab # [ 109.968845] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.193Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1157gitlab # [ 109.973285] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.198Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1158gitlab # [ 109.977472] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.202Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1159gitlab # [ 109.981701] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.206Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1160gitlab # [ 109.985904] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.210Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1161gitlab # [ 109.991309] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.216Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1162gitlab # [ 109.994848] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.219Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1163gitlab # [ 109.997870] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.222Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1164gitlab # [ 110.001505] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.226Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1165gitlab # [ 110.005475] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.230Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1166gitlab # [ 110.009935] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.234Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1167gitlab # [ 110.013963] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.238Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1168gitlab # [ 110.017970] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.242Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1169gitlab # [ 110.020831] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.245Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1170gitlab # [ 110.023702] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.248Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1171gitlab # [ 110.026411] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.251Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1172gitlab # [ 110.028845] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.253Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1173gitlab # [ 110.031126] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.256Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1174gitlab # [ 110.033513] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.258Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1175gitlab # [ 110.035948] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.260Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1176gitlab # [ 110.038456] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.263Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1177gitlab # [ 110.040963] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.265Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1178gitlab # [ 110.043631] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.268Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1179gitlab # [ 110.046482] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.271Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1180gitlab # [ 110.048766] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.273Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1181gitlab # [ 110.051544] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.276Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1182gitlab # [ 110.054038] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.279Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1183gitlab # [ 110.056697] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.281Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1184gitlab # [ 110.059363] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.284Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1185gitlab # [ 110.061497] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.286Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1186gitlab # [ 110.063595] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.288Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1187gitlab # [ 110.066292] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.291Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1188gitlab # [ 110.068375] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.293Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1189gitlab # [ 110.070776] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.295Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1190gitlab # [ 110.073334] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.298Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1191gitlab # [ 110.075540] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.300Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1192gitlab # [ 110.077849] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.302Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1193gitlab # [ 110.080927] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.305Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1194gitlab # [ 110.083771] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.308Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1195gitlab # [ 110.086202] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.311Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1196gitlab # [ 110.088410] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.313Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1197gitlab # [ 110.090795] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.315Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1198gitlab # [ 110.093371] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.318Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1199gitlab # [ 110.095776] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.320Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1200gitlab # [ 110.098700] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.323Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1201gitlab # [ 110.101861] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.326Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1202gitlab # [ 110.111784] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.336Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1203gitlab # [ 110.114916] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.339Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1204gitlab # [ 110.117713] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.342Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1205gitlab # [ 110.120567] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.345Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1206gitlab # [ 110.123244] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.348Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1207gitlab # [ 110.126096] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.351Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1208gitlab # [ 110.128490] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.353Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1209gitlab # [ 110.130962] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.355Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1210gitlab # [ 110.132969] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.357Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1211gitlab # [ 110.135401] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.360Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1212gitlab # [ 110.138305] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.363Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1213gitlab # [ 110.141198] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.366Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1214gitlab # [ 110.143713] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.368Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1215gitlab # [ 110.146739] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.371Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1216gitlab # [ 110.149051] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.374Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1217gitlab # [ 110.151295] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.376Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1218gitlab # [ 110.153721] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.378Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1219gitlab # [ 110.155632] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.380Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1220gitlab # [ 110.158201] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.383Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1221gitlab # [ 110.160281] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.385Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1222gitlab # [ 110.162783] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.387Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1223gitlab # [ 110.164701] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.389Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1224gitlab # [ 110.167357] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.392Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1225gitlab # [ 110.169833] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.394Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1226gitlab # [ 110.172252] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.397Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1227gitlab # [ 110.174386] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.399Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1228gitlab # [ 110.176623] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.401Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1229gitlab # [ 110.178733] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.403Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1230gitlab # [ 110.181197] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.406Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1231gitlab # [ 110.183518] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.408Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1232gitlab # [ 110.186547] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.411Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1233gitlab # [ 110.188743] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.413Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1234gitlab # [ 110.190943] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.415Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1235gitlab # [ 110.193354] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.418Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1236gitlab # [ 110.196130] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.421Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1237gitlab # [ 110.198454] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.423Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1238gitlab # [ 110.200711] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.425Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1239gitlab # [ 110.203336] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.428Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1240gitlab # [ 110.205431] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.430Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1241gitlab # [ 110.207728] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.432Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1242gitlab # [ 110.210019] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.435Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1243gitlab # [ 110.212385] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.437Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1244gitlab # [ 110.215405] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.440Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1245gitlab # [ 110.218133] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.443Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1246gitlab # [ 110.220287] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.445Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1247gitlab # [ 110.222660] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.447Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1248gitlab # [ 110.230763] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.455Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1249gitlab # [ 110.233349] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.458Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1250gitlab # [ 110.235637] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.460Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1251gitlab # [ 110.239101] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.464Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1252gitlab # [ 110.242084] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.467Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1253gitlab # [ 110.283542] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.508Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1254gitlab # [ 110.286962] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.511Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1255gitlab # [ 110.289740] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.514Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1256gitlab # [ 110.292685] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.517Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1257gitlab # [ 110.296782] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.521Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1258gitlab # [ 110.300267] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:51:40.525Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1259gitlab # [ 118.424649] gitlab-workhorse[825]: time="2026-11-10T09:51:48Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1260gitlab # [ 128.440283] gitlab-workhorse[825]: time="2026-11-10T09:51:58Z" level=error correlation_id= duration_ms=3 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1261gitlab # [ 133.318909] postgres[2137]: [2137] ERROR: relation "table_without_models" does not exist1262gitlab # [ 133.320152] postgres[2137]: [2137] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1263gitlab # [ 135.852602] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:06.076Z","message":"Booted Rails 7.2.3.1 application in production environment"}1264gitlab # [ 135.853911] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:06.078Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1265gitlab # [ 135.854687] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:06.078Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1266gitlab # [ 135.855596] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:06.078Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1267gitlab # [ 135.861757] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:06.086Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1268gitlab # [ 136.452776] postgres[2166]: [2166] ERROR: relation "table_without_models" does not exist1269gitlab # [ 136.455596] postgres[2166]: [2166] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1270gitlab # [ 137.514877] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:07.739Z","message":"Cleaning working queues"}1271gitlab # [ 137.517169] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:07.741Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"f3a8b114762a4e3ba00973ad","created_at":"2026-11-10T09:51:06.999Z","trace_propagation_headers":{"sentry-trace":"efb9896100e24024b9fa733183711334-a109125b4dfe41b6","baggage":"sentry-trace_id=efb9896100e24024b9fa733183711334,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d6d3308a12be8374222cbd780b8d0238","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T09:51:07.006Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-f3a8b114762a4e3ba00973ad: start","job_status":"start","queue_duration_s":60.734845,"scheduling_latency_s":60.734909}1272gitlab # [ 137.717050] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:07.941Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"f3a8b114762a4e3ba00973ad","created_at":"2026-11-10T09:51:06.999Z","trace_propagation_headers":{"sentry-trace":"efb9896100e24024b9fa733183711334-a109125b4dfe41b6","baggage":"sentry-trace_id=efb9896100e24024b9fa733183711334,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d6d3308a12be8374222cbd780b8d0238","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T09:51:07.006Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-f3a8b114762a4e3ba00973ad: done: 0.199777 sec","job_status":"done","queue_duration_s":60.734845,"scheduling_latency_s":60.734909,"redis_calls":31,"redis_duration_s":0.008722,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.005094,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002184,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09553476799999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.199777,"completed_at":"2026-11-10T09:52:07.941Z","load_balancing_strategy":"primary","db_duration_s":0.064055,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1273gitlab # [ 138.434776] gitlab-workhorse[825]: time="2026-11-10T09:52:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1274gitlab # [ 139.307414] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.531Z","pid":1623,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1275gitlab # [ 139.309924] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.531Z","pid":1623,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1276gitlab # [ 139.311306] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.531Z","pid":1623,"message":"! #\u003cThread:0x0000727916ab7f00 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1277gitlab # [ 139.312563] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.531Z","pid":1623,"message":"! #\u003cThread:0x0000727916ce98c8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1278gitlab # [ 139.313995] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.531Z","pid":1623,"message":"Use Ctrl-C to stop"}1279gitlab # [ 139.315315] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.532Z","pid":1623,"message":"Sending status to systemd every 1.0 sec"}1280gitlab # [ 139.668580] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.892Z","pid":1623,"message":"- Worker 1 (PID: 2233) booted in 0.3s, phase: 0"}1281gitlab # [ 139.670216] bundle[1623]: {"timestamp":"2026-11-10T09:52:09.895Z","pid":1623,"message":"- Worker 0 (PID: 2229) booted in 0.32s, phase: 0"}1282gitlab # [ 139.677908] systemd[1]: Started gitlab.service.1283gitlab # [ 139.679675] systemd[1]: Reached target Common target for all GitLab services..1284gitlab # [ 139.685768] systemd[1]: Starting setup-gitlab.service...1285gitlab # [ 139.701847] setup-gitlab-start[2257]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1286gitlab # [ 142.155208] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:52:12 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 24251287gitlab # [ 142.160885] setup-gitlab-start[2257]: + TOKEN=nixbot-test-token-of-20-chars1288gitlab # [ 142.162454] setup-gitlab-start[2257]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1289gitlab # [ 142.189454] runuser[2279]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1290gitlab # [ 142.589461] setup-gitlab-start[2280]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1291gitlab # [ 142.590673] setup-gitlab-start[2280]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1292gitlab # [ 142.591699] setup-gitlab-start[2280]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1293gitlab # [ 143.192174] setup-gitlab-start[2280]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1294gitlab # [ 143.193062] setup-gitlab-start[2280]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1295gitlab # [ 143.194360] setup-gitlab-start[2280]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1296gitlab # [ 147.887141] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3174deff90ec3b9ad11a4332","created_at":"2026-11-10T09:52:18.073Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1038f6101c60b48f6c79cda9a6f3bec2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.107Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3174deff90ec3b9ad11a4332: start","job_status":"start","queue_duration_s":0.002794,"scheduling_latency_s":0.002845}1297gitlab # [ 148.012213] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3174deff90ec3b9ad11a4332","created_at":"2026-11-10T09:52:18.073Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1038f6101c60b48f6c79cda9a6f3bec2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.107Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3174deff90ec3b9ad11a4332: done: 0.127007 sec","job_status":"done","queue_duration_s":0.002794,"scheduling_latency_s":0.002845,"redis_calls":12,"redis_duration_s":0.014519999999999998,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004748,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008006,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001766,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000586388,"duration_s":0.127007,"completed_at":"2026-11-10T09:52:18.236Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 148.082278] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"784564f5cba9ae0a3d7a8aa2","created_at":"2026-11-10T09:52:18.159Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7b3e6d564e33b9b19a6afcd4274e0c0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.177Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-784564f5cba9ae0a3d7a8aa2: start","job_status":"start","queue_duration_s":0.074499,"scheduling_latency_s":0.074519}1299gitlab # [ 148.152595] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"784564f5cba9ae0a3d7a8aa2","created_at":"2026-11-10T09:52:18.159Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7b3e6d564e33b9b19a6afcd4274e0c0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.177Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-784564f5cba9ae0a3d7a8aa2: done: 0.124786 sec","job_status":"done","queue_duration_s":0.074499,"scheduling_latency_s":0.074519,"redis_calls":14,"redis_duration_s":0.00594,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002358,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000361778,"exclusive_lock_hold_duration_s":0.097598717,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.124786,"completed_at":"2026-11-10T09:52:18.376Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 148.190993] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ceb533e0e132b2683765b3e1","created_at":"2026-11-10T09:52:18.183Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3aa0086df73c19daa13e844d8df5ad57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.191Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ceb533e0e132b2683765b3e1: start","job_status":"start","queue_duration_s":0.18907,"scheduling_latency_s":0.189088}1301gitlab # [ 148.191723] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.402Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1302gitlab # [ 148.192229] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ceb533e0e132b2683765b3e1","created_at":"2026-11-10T09:52:18.183Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3aa0086df73c19daa13e844d8df5ad57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.191Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ceb533e0e132b2683765b3e1: done: 0.030305 sec","job_status":"done","queue_duration_s":0.18907,"scheduling_latency_s":0.189088,"redis_calls":10,"redis_duration_s":0.006122,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001383,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000246121,"duration_s":0.030305,"completed_at":"2026-11-10T09:52:18.410Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 148.252874] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bd8ac1710f16854641c74c56","created_at":"2026-11-10T09:52:18.219Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8f05d137cd1fbc8433dca54f06f475b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.230Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bd8ac1710f16854641c74c56: start","job_status":"start","queue_duration_s":0.181225,"scheduling_latency_s":0.181239}1304gitlab # [ 148.253612] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bd8ac1710f16854641c74c56","created_at":"2026-11-10T09:52:18.219Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8f05d137cd1fbc8433dca54f06f475b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.230Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bd8ac1710f16854641c74c56: done: 0.031515 sec","job_status":"done","queue_duration_s":0.181225,"scheduling_latency_s":0.181239,"redis_calls":10,"redis_duration_s":0.005521,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000259251,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.031515,"completed_at":"2026-11-10T09:52:18.443Z","load_balancing_strategy":"primary","db_duration_s":0.01232,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 148.306764] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f9485cee292aa89c5bd38d2","created_at":"2026-11-10T09:52:18.253Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.288Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f9485cee292aa89c5bd38d2: start","job_status":"start","queue_duration_s":0.15572,"scheduling_latency_s":0.155735}1306gitlab # [ 148.341580] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f9485cee292aa89c5bd38d2","created_at":"2026-11-10T09:52:18.253Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.288Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f9485cee292aa89c5bd38d2: done: 0.12164 sec","job_status":"done","queue_duration_s":0.15572,"scheduling_latency_s":0.155735,"redis_calls":71,"redis_duration_s":0.056738,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001266,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.045547,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009014,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000397258,"duration_s":0.12164,"completed_at":"2026-11-10T09:52:18.565Z","load_balancing_strategy":"primary","db_duration_s":0.001215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 148.384131] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"037deca7be8c25044abc326b","created_at":"2026-11-10T09:52:18.302Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49550b875513357c67619534f3ba331f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.323Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-037deca7be8c25044abc326b: start","job_status":"start","queue_duration_s":0.284894,"scheduling_latency_s":0.284927}1308gitlab # [ 148.429386] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"037deca7be8c25044abc326b","created_at":"2026-11-10T09:52:18.302Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49550b875513357c67619534f3ba331f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.323Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-037deca7be8c25044abc326b: done: 0.045468 sec","job_status":"done","queue_duration_s":0.284894,"scheduling_latency_s":0.284927,"redis_calls":13,"redis_duration_s":0.011207,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001014,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007681,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002512,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009957132,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.045468,"completed_at":"2026-11-10T09:52:18.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004767,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1309gitlab # [ 148.439270] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.664Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"e2ec7f1bec95a4127f8d4ceb","created_at":"2026-11-10T09:52:18.334Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"7642246113ebb6b6a44557a4ebede153","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.347Z","job_size_bytes":3,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-e2ec7f1bec95a4127f8d4ceb: start","job_status":"start","queue_duration_s":0.316761,"scheduling_latency_s":0.316781}1310gitlab # [ 148.456411] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.680Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"e2ec7f1bec95a4127f8d4ceb","created_at":"2026-11-10T09:52:18.334Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"7642246113ebb6b6a44557a4ebede153","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.347Z","job_size_bytes":3,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-e2ec7f1bec95a4127f8d4ceb: done: 0.016898 sec","job_status":"done","queue_duration_s":0.316761,"scheduling_latency_s":0.316781,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238019,"duration_s":0.016898,"completed_at":"2026-11-10T09:52:18.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 148.504104] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:18.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c19a997c990ce9402b5ac092","created_at":"2026-11-10T09:52:18.360Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c3b8490ed4404cb992386fc87792a73","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.370Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c19a997c990ce9402b5ac092: start","job_status":"start","queue_duration_s":0.311414,"scheduling_latency_s":0.311427}1312gitlab # [ 148.978756] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c19a997c990ce9402b5ac092","created_at":"2026-11-10T09:52:18.360Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8c3b8490ed4404cb992386fc87792a73","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.370Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c19a997c990ce9402b5ac092: done: 0.519468 sec","job_status":"done","queue_duration_s":0.311414,"scheduling_latency_s":0.311427,"redis_calls":565,"redis_duration_s":0.23367300000000002,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.232355,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.324065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000619073,"duration_s":0.519468,"completed_at":"2026-11-10T09:52:19.201Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 148.981143] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"78ca98eb6917d3023e683cdc","created_at":"2026-11-10T09:52:18.381Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc3e2f6c3d10b5f7a47023bb01085243","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.390Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-78ca98eb6917d3023e683cdc: start","job_status":"start","queue_duration_s":0.813665,"scheduling_latency_s":0.813694}1314gitlab # [ 149.183270] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"78ca98eb6917d3023e683cdc","created_at":"2026-11-10T09:52:18.381Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc3e2f6c3d10b5f7a47023bb01085243","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.390Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-78ca98eb6917d3023e683cdc: done: 0.204144 sec","job_status":"done","queue_duration_s":0.813665,"scheduling_latency_s":0.813694,"redis_calls":14,"redis_duration_s":0.003875,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000972,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000716292,"exclusive_lock_hold_duration_s":0.179640988,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.204144,"completed_at":"2026-11-10T09:52:19.407Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.074716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 149.186581] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7feda6ef134634ea492a1b66","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7feda6ef134634ea492a1b66: start","job_status":"start","queue_duration_s":0.902301,"scheduling_latency_s":0.902316}1316gitlab # [ 149.211571] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7feda6ef134634ea492a1b66","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7feda6ef134634ea492a1b66: done: 0.019856 sec","job_status":"done","queue_duration_s":0.902301,"scheduling_latency_s":0.902316,"redis_calls":16,"redis_duration_s":0.0026030000000000003,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001561,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000209523,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019856,"completed_at":"2026-11-10T09:52:19.429Z","load_balancing_strategy":"primary","db_duration_s":0.000577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 149.278281] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97a24ad9336a6e3dc5cacc3e","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97a24ad9336a6e3dc5cacc3e: start","job_status":"start","queue_duration_s":0.923125,"scheduling_latency_s":0.923165}1318gitlab # [ 149.323701] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97a24ad9336a6e3dc5cacc3e","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97a24ad9336a6e3dc5cacc3e: done: 0.013724 sec","job_status":"done","queue_duration_s":0.923125,"scheduling_latency_s":0.923165,"redis_calls":13,"redis_duration_s":0.002081,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013724,"completed_at":"2026-11-10T09:52:19.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 149.325278] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e23173a6808ccee5f122d2c","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e23173a6808ccee5f122d2c: start","job_status":"start","queue_duration_s":0.938017,"scheduling_latency_s":0.938032}1320gitlab # [ 149.368616] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e23173a6808ccee5f122d2c","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e23173a6808ccee5f122d2c: done: 0.011866 sec","job_status":"done","queue_duration_s":0.938017,"scheduling_latency_s":0.938032,"redis_calls":13,"redis_duration_s":0.002914,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011866,"completed_at":"2026-11-10T09:52:19.456Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 149.373526] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45c268889e27c05e2453dc62","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45c268889e27c05e2453dc62: start","job_status":"start","queue_duration_s":0.95144,"scheduling_latency_s":0.951454}1322gitlab # [ 149.428864] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45c268889e27c05e2453dc62","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45c268889e27c05e2453dc62: done: 0.01278 sec","job_status":"done","queue_duration_s":0.95144,"scheduling_latency_s":0.951454,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01278,"completed_at":"2026-11-10T09:52:19.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 149.432664] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84a0e95055fa4c411e95b19","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84a0e95055fa4c411e95b19: start","job_status":"start","queue_duration_s":0.965641,"scheduling_latency_s":0.965657}1324gitlab # [ 149.445722] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84a0e95055fa4c411e95b19","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84a0e95055fa4c411e95b19: done: 0.01276 sec","job_status":"done","queue_duration_s":0.965641,"scheduling_latency_s":0.965657,"redis_calls":13,"redis_duration_s":0.00314,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01276,"completed_at":"2026-11-10T09:52:19.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 149.502451] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f722c29f3cc819d32dbf4bf","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f722c29f3cc819d32dbf4bf: start","job_status":"start","queue_duration_s":0.979809,"scheduling_latency_s":0.979828}1326gitlab # [ 149.503343] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f722c29f3cc819d32dbf4bf","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f722c29f3cc819d32dbf4bf: done: 0.014691 sec","job_status":"done","queue_duration_s":0.979809,"scheduling_latency_s":0.979828,"redis_calls":13,"redis_duration_s":0.003711,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002482,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014691,"completed_at":"2026-11-10T09:52:19.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 149.560740] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efbcb89710912e5dc8149b67","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efbcb89710912e5dc8149b67: start","job_status":"start","queue_duration_s":0.996064,"scheduling_latency_s":0.996082}1328gitlab # [ 149.561558] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efbcb89710912e5dc8149b67","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efbcb89710912e5dc8149b67: done: 0.017451 sec","job_status":"done","queue_duration_s":0.996064,"scheduling_latency_s":0.996082,"redis_calls":13,"redis_duration_s":0.003725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002631,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017451,"completed_at":"2026-11-10T09:52:19.520Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 149.625245] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cd7e367c012c64924e23c7d","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cd7e367c012c64924e23c7d: start","job_status":"start","queue_duration_s":1.015497,"scheduling_latency_s":1.015511}1330gitlab # [ 149.626296] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cd7e367c012c64924e23c7d","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cd7e367c012c64924e23c7d: done: 0.013951 sec","job_status":"done","queue_duration_s":1.015497,"scheduling_latency_s":1.015511,"redis_calls":13,"redis_duration_s":0.0032069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013951,"completed_at":"2026-11-10T09:52:19.536Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 149.687976] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"780ef02c4e7bc22810a95609","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-780ef02c4e7bc22810a95609: start","job_status":"start","queue_duration_s":1.030569,"scheduling_latency_s":1.030583}1332gitlab # [ 149.689088] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"780ef02c4e7bc22810a95609","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-780ef02c4e7bc22810a95609: done: 0.012509 sec","job_status":"done","queue_duration_s":1.030569,"scheduling_latency_s":1.030583,"redis_calls":13,"redis_duration_s":0.00263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012509,"completed_at":"2026-11-10T09:52:19.550Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 149.744921] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec6cc3835b5469637fd1c9fa","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec6cc3835b5469637fd1c9fa: start","job_status":"start","queue_duration_s":1.044032,"scheduling_latency_s":1.044046}1334gitlab # [ 149.745901] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec6cc3835b5469637fd1c9fa","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec6cc3835b5469637fd1c9fa: done: 0.011774 sec","job_status":"done","queue_duration_s":1.044032,"scheduling_latency_s":1.044046,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011774,"completed_at":"2026-11-10T09:52:19.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 149.803041] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aec8c50d1ee06f5d7abaa51","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aec8c50d1ee06f5d7abaa51: start","job_status":"start","queue_duration_s":1.058461,"scheduling_latency_s":1.058516}1336gitlab # [ 149.803686] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aec8c50d1ee06f5d7abaa51","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aec8c50d1ee06f5d7abaa51: done: 0.018985 sec","job_status":"done","queue_duration_s":1.058461,"scheduling_latency_s":1.058516,"redis_calls":13,"redis_duration_s":0.003005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018985,"completed_at":"2026-11-10T09:52:19.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 149.846799] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f68cb02a35a459cd6c480dc","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f68cb02a35a459cd6c480dc: start","job_status":"start","queue_duration_s":1.078882,"scheduling_latency_s":1.078909}1338gitlab # [ 149.847732] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f68cb02a35a459cd6c480dc","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f68cb02a35a459cd6c480dc: done: 0.020852 sec","job_status":"done","queue_duration_s":1.078882,"scheduling_latency_s":1.078909,"redis_calls":13,"redis_duration_s":0.003367,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020852,"completed_at":"2026-11-10T09:52:19.606Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 149.859817] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c97b1c043bfef4dbbb22594f","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c97b1c043bfef4dbbb22594f: start","job_status":"start","queue_duration_s":1.102029,"scheduling_latency_s":1.10206}1340gitlab # [ 149.871052] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c97b1c043bfef4dbbb22594f","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c97b1c043bfef4dbbb22594f: done: 0.023402 sec","job_status":"done","queue_duration_s":1.102029,"scheduling_latency_s":1.10206,"redis_calls":13,"redis_duration_s":0.004388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023402,"completed_at":"2026-11-10T09:52:19.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 149.913981] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6063c63f90dbeae48521b03d","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6063c63f90dbeae48521b03d: start","job_status":"start","queue_duration_s":1.12796,"scheduling_latency_s":1.127994}1342gitlab # [ 149.916996] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6063c63f90dbeae48521b03d","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6063c63f90dbeae48521b03d: done: 0.028424 sec","job_status":"done","queue_duration_s":1.12796,"scheduling_latency_s":1.127994,"redis_calls":13,"redis_duration_s":0.004361,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028424,"completed_at":"2026-11-10T09:52:19.663Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 149.960719] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e7f3386b3318470e66bc5b2","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e7f3386b3318470e66bc5b2: start","job_status":"start","queue_duration_s":1.159655,"scheduling_latency_s":1.159705}1344gitlab # [ 150.000269] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e7f3386b3318470e66bc5b2","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e7f3386b3318470e66bc5b2: done: 0.018687 sec","job_status":"done","queue_duration_s":1.159655,"scheduling_latency_s":1.159705,"redis_calls":13,"redis_duration_s":0.0022459999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018687,"completed_at":"2026-11-10T09:52:19.685Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 150.001554] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a849adaf1f29453eb296f9b","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a849adaf1f29453eb296f9b: start","job_status":"start","queue_duration_s":1.179361,"scheduling_latency_s":1.179376}1346gitlab # [ 150.008713] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a849adaf1f29453eb296f9b","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a849adaf1f29453eb296f9b: done: 0.012936 sec","job_status":"done","queue_duration_s":1.179361,"scheduling_latency_s":1.179376,"redis_calls":13,"redis_duration_s":0.00194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012936,"completed_at":"2026-11-10T09:52:19.699Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 150.056034] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1eb8a964181009aad91a7e77","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1eb8a964181009aad91a7e77: start","job_status":"start","queue_duration_s":1.193497,"scheduling_latency_s":1.193511}1348gitlab # [ 150.056383] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1eb8a964181009aad91a7e77","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1eb8a964181009aad91a7e77: done: 0.011479 sec","job_status":"done","queue_duration_s":1.193497,"scheduling_latency_s":1.193511,"redis_calls":13,"redis_duration_s":0.00197,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011479,"completed_at":"2026-11-10T09:52:19.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 150.090686] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5533fd0a68f499f1cac8d690","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5533fd0a68f499f1cac8d690: start","job_status":"start","queue_duration_s":1.206529,"scheduling_latency_s":1.206544}1350gitlab # [ 150.101885] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5533fd0a68f499f1cac8d690","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5533fd0a68f499f1cac8d690: done: 0.022042 sec","job_status":"done","queue_duration_s":1.206529,"scheduling_latency_s":1.206544,"redis_calls":13,"redis_duration_s":0.004922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022042,"completed_at":"2026-11-10T09:52:19.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 150.158784] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ac48de55f66bb3033aa3c01","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ac48de55f66bb3033aa3c01: start","job_status":"start","queue_duration_s":1.230638,"scheduling_latency_s":1.230663}1352gitlab # [ 150.159664] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ac48de55f66bb3033aa3c01","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ac48de55f66bb3033aa3c01: done: 0.019147 sec","job_status":"done","queue_duration_s":1.230638,"scheduling_latency_s":1.230663,"redis_calls":13,"redis_duration_s":0.0024749999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001699,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019147,"completed_at":"2026-11-10T09:52:19.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 150.219139] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d49b314e9e95f6b457d7f4ae","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d49b314e9e95f6b457d7f4ae: start","job_status":"start","queue_duration_s":1.251544,"scheduling_latency_s":1.25157}1354gitlab # [ 150.219929] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d49b314e9e95f6b457d7f4ae","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d49b314e9e95f6b457d7f4ae: done: 0.016679 sec","job_status":"done","queue_duration_s":1.251544,"scheduling_latency_s":1.25157,"redis_calls":13,"redis_duration_s":0.0023020000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001492,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016679,"completed_at":"2026-11-10T09:52:19.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 150.260220] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1537ebede1a479e9132eb786","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1537ebede1a479e9132eb786: start","job_status":"start","queue_duration_s":1.269581,"scheduling_latency_s":1.269613}1356gitlab # [ 150.260936] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1537ebede1a479e9132eb786","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1537ebede1a479e9132eb786: done: 0.012199 sec","job_status":"done","queue_duration_s":1.269581,"scheduling_latency_s":1.269613,"redis_calls":13,"redis_duration_s":0.001702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012199,"completed_at":"2026-11-10T09:52:19.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 150.311616] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1de7faaef4aaf78bfed6324","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1de7faaef4aaf78bfed6324: start","job_status":"start","queue_duration_s":1.283287,"scheduling_latency_s":1.283305}1358gitlab # [ 150.312478] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1de7faaef4aaf78bfed6324","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1de7faaef4aaf78bfed6324: done: 0.016258 sec","job_status":"done","queue_duration_s":1.283287,"scheduling_latency_s":1.283305,"redis_calls":13,"redis_duration_s":0.002537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001711,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016258,"completed_at":"2026-11-10T09:52:19.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 150.359712] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ba999a929fac4dcf998b1e9","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ba999a929fac4dcf998b1e9: start","job_status":"start","queue_duration_s":1.300664,"scheduling_latency_s":1.300681}1360gitlab # [ 150.360637] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ba999a929fac4dcf998b1e9","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ba999a929fac4dcf998b1e9: done: 0.019969 sec","job_status":"done","queue_duration_s":1.300664,"scheduling_latency_s":1.300681,"redis_calls":13,"redis_duration_s":0.0035529999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019969,"completed_at":"2026-11-10T09:52:19.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 150.406200] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feff68e93e2f7c132c5975a8","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feff68e93e2f7c132c5975a8: start","job_status":"start","queue_duration_s":1.322159,"scheduling_latency_s":1.322182}1362gitlab # [ 150.406926] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feff68e93e2f7c132c5975a8","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feff68e93e2f7c132c5975a8: done: 0.023496 sec","job_status":"done","queue_duration_s":1.322159,"scheduling_latency_s":1.322182,"redis_calls":13,"redis_duration_s":0.002763,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001754,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023496,"completed_at":"2026-11-10T09:52:19.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 150.417775] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd7589a2646b55b06d8667f2","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd7589a2646b55b06d8667f2: start","job_status":"start","queue_duration_s":1.347956,"scheduling_latency_s":1.347996}1364gitlab # [ 150.429259] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd7589a2646b55b06d8667f2","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd7589a2646b55b06d8667f2: done: 0.02626 sec","job_status":"done","queue_duration_s":1.347956,"scheduling_latency_s":1.347996,"redis_calls":13,"redis_duration_s":0.0032129999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02626,"completed_at":"2026-11-10T09:52:19.881Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 150.488801] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6b5fa42ff7aa2f47f0a7157","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6b5fa42ff7aa2f47f0a7157: start","job_status":"start","queue_duration_s":1.375864,"scheduling_latency_s":1.375879}1366gitlab # [ 150.489679] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6b5fa42ff7aa2f47f0a7157","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6b5fa42ff7aa2f47f0a7157: done: 0.025894 sec","job_status":"done","queue_duration_s":1.375864,"scheduling_latency_s":1.375879,"redis_calls":13,"redis_duration_s":0.003156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025894,"completed_at":"2026-11-10T09:52:19.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 150.538735] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85eaa376154e86f8320a1d7e","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85eaa376154e86f8320a1d7e: start","job_status":"start","queue_duration_s":1.403752,"scheduling_latency_s":1.403769}1368gitlab # [ 150.539588] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85eaa376154e86f8320a1d7e","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85eaa376154e86f8320a1d7e: done: 0.025517 sec","job_status":"done","queue_duration_s":1.403752,"scheduling_latency_s":1.403769,"redis_calls":13,"redis_duration_s":0.003222,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025517,"completed_at":"2026-11-10T09:52:19.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 150.590612] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0314553da0633a603233874","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0314553da0633a603233874: start","job_status":"start","queue_duration_s":1.431101,"scheduling_latency_s":1.431175}1370gitlab # [ 150.591283] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0314553da0633a603233874","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0314553da0633a603233874: done: 0.020736 sec","job_status":"done","queue_duration_s":1.431101,"scheduling_latency_s":1.431175,"redis_calls":13,"redis_duration_s":0.002724,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020736,"completed_at":"2026-11-10T09:52:19.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 150.599937] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0816bc95d44d80508b9a44a9","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0816bc95d44d80508b9a44a9: start","job_status":"start","queue_duration_s":1.453504,"scheduling_latency_s":1.453544}1372gitlab # [ 150.646911] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0816bc95d44d80508b9a44a9","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0816bc95d44d80508b9a44a9: done: 0.021859 sec","job_status":"done","queue_duration_s":1.453504,"scheduling_latency_s":1.453544,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021859,"completed_at":"2026-11-10T09:52:19.982Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 150.647906] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:19.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72c6839968e251ff81890411","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72c6839968e251ff81890411: start","job_status":"start","queue_duration_s":1.477363,"scheduling_latency_s":1.4774}1374gitlab # [ 150.686573] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72c6839968e251ff81890411","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72c6839968e251ff81890411: done: 0.020724 sec","job_status":"done","queue_duration_s":1.477363,"scheduling_latency_s":1.4774,"redis_calls":13,"redis_duration_s":0.0029790000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020724,"completed_at":"2026-11-10T09:52:20.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 150.725596] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"455b900478aae3f88b4a4732","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-455b900478aae3f88b4a4732: start","job_status":"start","queue_duration_s":1.499162,"scheduling_latency_s":1.499182}1376gitlab # [ 150.726305] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"455b900478aae3f88b4a4732","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-455b900478aae3f88b4a4732: done: 0.010196 sec","job_status":"done","queue_duration_s":1.499162,"scheduling_latency_s":1.499182,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010196,"completed_at":"2026-11-10T09:52:20.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 150.734244] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6087e31375e016b479ba4428","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6087e31375e016b479ba4428: start","job_status":"start","queue_duration_s":1.510313,"scheduling_latency_s":1.510326}1378gitlab # [ 150.745777] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6087e31375e016b479ba4428","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6087e31375e016b479ba4428: done: 0.011609 sec","job_status":"done","queue_duration_s":1.510313,"scheduling_latency_s":1.510326,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011609,"completed_at":"2026-11-10T09:52:20.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 150.803567] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"658443fee3d402c8df2eadba","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-658443fee3d402c8df2eadba: start","job_status":"start","queue_duration_s":1.52305,"scheduling_latency_s":1.523065}1380gitlab # [ 150.804333] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"658443fee3d402c8df2eadba","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-658443fee3d402c8df2eadba: done: 0.00975 sec","job_status":"done","queue_duration_s":1.52305,"scheduling_latency_s":1.523065,"redis_calls":13,"redis_duration_s":0.0015570000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00975,"completed_at":"2026-11-10T09:52:20.039Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 150.825135] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f92ba444e543fe387cdd131","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f92ba444e543fe387cdd131: start","job_status":"start","queue_duration_s":1.533813,"scheduling_latency_s":1.533826}1382gitlab # [ 150.836210] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f92ba444e543fe387cdd131","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f92ba444e543fe387cdd131: done: 0.009767 sec","job_status":"done","queue_duration_s":1.533813,"scheduling_latency_s":1.533826,"redis_calls":13,"redis_duration_s":0.001576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009767,"completed_at":"2026-11-10T09:52:20.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 150.885978] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"161c166a8420b03594351a63","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-161c166a8420b03594351a63: start","job_status":"start","queue_duration_s":1.544596,"scheduling_latency_s":1.544611}1384gitlab # [ 150.886729] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"161c166a8420b03594351a63","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-161c166a8420b03594351a63: done: 0.009635 sec","job_status":"done","queue_duration_s":1.544596,"scheduling_latency_s":1.544611,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009635,"completed_at":"2026-11-10T09:52:20.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 150.916388] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3327adee8a6bf5e41460bf95","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3327adee8a6bf5e41460bf95: start","job_status":"start","queue_duration_s":1.555085,"scheduling_latency_s":1.555098}1386gitlab # [ 150.927629] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3327adee8a6bf5e41460bf95","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3327adee8a6bf5e41460bf95: done: 0.010208 sec","job_status":"done","queue_duration_s":1.555085,"scheduling_latency_s":1.555098,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010208,"completed_at":"2026-11-10T09:52:20.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 150.966665] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77f121261f768f4111cf67c9","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77f121261f768f4111cf67c9: start","job_status":"start","queue_duration_s":1.566193,"scheduling_latency_s":1.566207}1388gitlab # [ 150.972208] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77f121261f768f4111cf67c9","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77f121261f768f4111cf67c9: done: 0.010257 sec","job_status":"done","queue_duration_s":1.566193,"scheduling_latency_s":1.566207,"redis_calls":13,"redis_duration_s":0.0016059999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010257,"completed_at":"2026-11-10T09:52:20.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 151.014431] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be59d344c0c6db0149a96a18","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be59d344c0c6db0149a96a18: start","job_status":"start","queue_duration_s":1.577352,"scheduling_latency_s":1.577366}1390gitlab # [ 151.056293] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be59d344c0c6db0149a96a18","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be59d344c0c6db0149a96a18: done: 0.009905 sec","job_status":"done","queue_duration_s":1.577352,"scheduling_latency_s":1.577366,"redis_calls":13,"redis_duration_s":0.001599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009905,"completed_at":"2026-11-10T09:52:20.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 151.057223] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f90c31323ab5496b2f0548d","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f90c31323ab5496b2f0548d: start","job_status":"start","queue_duration_s":1.588082,"scheduling_latency_s":1.588095}1392gitlab # [ 151.096350] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f90c31323ab5496b2f0548d","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f90c31323ab5496b2f0548d: done: 0.010116 sec","job_status":"done","queue_duration_s":1.588082,"scheduling_latency_s":1.588095,"redis_calls":13,"redis_duration_s":0.00159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010116,"completed_at":"2026-11-10T09:52:20.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 151.137567] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8e859e673729e1ee3b1eda3","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8e859e673729e1ee3b1eda3: start","job_status":"start","queue_duration_s":1.599055,"scheduling_latency_s":1.599069}1394gitlab # [ 151.138320] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8e859e673729e1ee3b1eda3","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8e859e673729e1ee3b1eda3: done: 0.009612 sec","job_status":"done","queue_duration_s":1.599055,"scheduling_latency_s":1.599069,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009612,"completed_at":"2026-11-10T09:52:20.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 151.180425] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec201164f6e3eba7efdae733","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec201164f6e3eba7efdae733: start","job_status":"start","queue_duration_s":1.609931,"scheduling_latency_s":1.609945}1396gitlab # [ 151.181096] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec201164f6e3eba7efdae733","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec201164f6e3eba7efdae733: done: 0.009679 sec","job_status":"done","queue_duration_s":1.609931,"scheduling_latency_s":1.609945,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009679,"completed_at":"2026-11-10T09:52:20.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 151.193039] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"868b1317a4717ae5dc74c463","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-868b1317a4717ae5dc74c463: start","job_status":"start","queue_duration_s":1.620637,"scheduling_latency_s":1.620652}1398gitlab # [ 151.240554] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"868b1317a4717ae5dc74c463","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-868b1317a4717ae5dc74c463: done: 0.009869 sec","job_status":"done","queue_duration_s":1.620637,"scheduling_latency_s":1.620652,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009869,"completed_at":"2026-11-10T09:52:20.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 151.241591] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de087512b2bb8a0319921d5","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de087512b2bb8a0319921d5: start","job_status":"start","queue_duration_s":1.631661,"scheduling_latency_s":1.631675}1400gitlab # [ 151.250099] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de087512b2bb8a0319921d5","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de087512b2bb8a0319921d5: done: 0.009933 sec","job_status":"done","queue_duration_s":1.631661,"scheduling_latency_s":1.631675,"redis_calls":13,"redis_duration_s":0.0017169999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009933,"completed_at":"2026-11-10T09:52:20.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 151.290116] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc7e1cbbef557a8548fae8ad","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc7e1cbbef557a8548fae8ad: start","job_status":"start","queue_duration_s":1.642645,"scheduling_latency_s":1.64266}1402gitlab # [ 151.295847] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc7e1cbbef557a8548fae8ad","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc7e1cbbef557a8548fae8ad: done: 0.010955 sec","job_status":"done","queue_duration_s":1.642645,"scheduling_latency_s":1.64266,"redis_calls":13,"redis_duration_s":0.0016809999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010955,"completed_at":"2026-11-10T09:52:20.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 151.339852] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfd7491dea6e5d87a26a7f29","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfd7491dea6e5d87a26a7f29: start","job_status":"start","queue_duration_s":1.654645,"scheduling_latency_s":1.654658}1404gitlab # [ 151.341928] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfd7491dea6e5d87a26a7f29","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfd7491dea6e5d87a26a7f29: done: 0.009895 sec","job_status":"done","queue_duration_s":1.654645,"scheduling_latency_s":1.654658,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009895,"completed_at":"2026-11-10T09:52:20.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 151.392366] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9a0b67cc3bc37768e4d964a","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9a0b67cc3bc37768e4d964a: start","job_status":"start","queue_duration_s":1.665699,"scheduling_latency_s":1.665713}1406gitlab # [ 151.392958] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9a0b67cc3bc37768e4d964a","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9a0b67cc3bc37768e4d964a: done: 0.009771 sec","job_status":"done","queue_duration_s":1.665699,"scheduling_latency_s":1.665713,"redis_calls":13,"redis_duration_s":0.001718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009771,"completed_at":"2026-11-10T09:52:20.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 151.438708] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98b9fa705a85748a0f6b44b4","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98b9fa705a85748a0f6b44b4: start","job_status":"start","queue_duration_s":1.676349,"scheduling_latency_s":1.676364}1408gitlab # [ 151.439616] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98b9fa705a85748a0f6b44b4","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98b9fa705a85748a0f6b44b4: done: 0.010718 sec","job_status":"done","queue_duration_s":1.676349,"scheduling_latency_s":1.676364,"redis_calls":13,"redis_duration_s":0.002326,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010718,"completed_at":"2026-11-10T09:52:20.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 151.482696] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91f2364bb9b4bd4719c40877","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91f2364bb9b4bd4719c40877: start","job_status":"start","queue_duration_s":1.687963,"scheduling_latency_s":1.687977}1410gitlab # [ 151.483477] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91f2364bb9b4bd4719c40877","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91f2364bb9b4bd4719c40877: done: 0.010104 sec","job_status":"done","queue_duration_s":1.687963,"scheduling_latency_s":1.687977,"redis_calls":13,"redis_duration_s":0.001862,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010104,"completed_at":"2026-11-10T09:52:20.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 151.534110] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2474daa80a42e08d32d49ce0","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2474daa80a42e08d32d49ce0: start","job_status":"start","queue_duration_s":1.699008,"scheduling_latency_s":1.699023}1412gitlab # [ 151.534825] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2474daa80a42e08d32d49ce0","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2474daa80a42e08d32d49ce0: done: 0.010255 sec","job_status":"done","queue_duration_s":1.699008,"scheduling_latency_s":1.699023,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010255,"completed_at":"2026-11-10T09:52:20.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 151.573751] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0676e8c0b8c6b89449c35f2f","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0676e8c0b8c6b89449c35f2f: start","job_status":"start","queue_duration_s":1.710363,"scheduling_latency_s":1.710377}1414gitlab # [ 151.574662] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0676e8c0b8c6b89449c35f2f","created_at":"2026-11-10T09:52:18.457Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-168d9d1e3db54c3c","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b8988c3e714d4f6655c36f13a85f047e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.506Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0676e8c0b8c6b89449c35f2f: done: 0.010065 sec","job_status":"done","queue_duration_s":1.710363,"scheduling_latency_s":1.710377,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010065,"completed_at":"2026-11-10T09:52:20.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 151.614046] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.228Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"31d07fd7218b5399ee2c75f2","created_at":"2026-11-10T09:52:18.403Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3d6fca03cd0c417d37f1310a7bcb21d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.555Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-31d07fd7218b5399ee2c75f2: start","job_status":"start","queue_duration_s":1.673033,"scheduling_latency_s":1.673048}1416gitlab # [ 151.615823] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"31d07fd7218b5399ee2c75f2","created_at":"2026-11-10T09:52:18.403Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3d6fca03cd0c417d37f1310a7bcb21d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.555Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-31d07fd7218b5399ee2c75f2: done: 0.020029 sec","job_status":"done","queue_duration_s":1.673033,"scheduling_latency_s":1.673048,"redis_calls":12,"redis_duration_s":0.001663,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000315,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00033887,"exclusive_lock_hold_duration_s":0.0063164449999999995,"duration_s":0.020029,"completed_at":"2026-11-10T09:52:20.248Z","load_balancing_strategy":"primary","db_duration_s":0.002442,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 151.660800] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.249Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7bd5b8e44a5b06cd9cb4a650","created_at":"2026-11-10T09:52:18.604Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51b70016fe2bd6ccb1b6fc9ac73a812","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.612Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7bd5b8e44a5b06cd9cb4a650: start","job_status":"start","queue_duration_s":1.637193,"scheduling_latency_s":1.637207}1418gitlab # [ 151.664636] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7bd5b8e44a5b06cd9cb4a650","created_at":"2026-11-10T09:52:18.604Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51b70016fe2bd6ccb1b6fc9ac73a812","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.612Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7bd5b8e44a5b06cd9cb4a650: done: 0.083493 sec","job_status":"done","queue_duration_s":1.637193,"scheduling_latency_s":1.637207,"redis_calls":12,"redis_duration_s":0.002102,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000565,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230755,"duration_s":0.083493,"completed_at":"2026-11-10T09:52:20.332Z","load_balancing_strategy":"primary","db_duration_s":0.000535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 151.712704] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.333Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"01c2c8531d3d07af5e7ca493","created_at":"2026-11-10T09:52:18.622Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84db746d60415e76b8862120af33dbe2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.632Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-01c2c8531d3d07af5e7ca493: start","job_status":"start","queue_duration_s":1.701372,"scheduling_latency_s":1.701388}1420gitlab # [ 151.715884] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.341Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1421gitlab # [ 151.718914] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"01c2c8531d3d07af5e7ca493","created_at":"2026-11-10T09:52:18.622Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84db746d60415e76b8862120af33dbe2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.632Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-01c2c8531d3d07af5e7ca493: done: 0.012078 sec","job_status":"done","queue_duration_s":1.701372,"scheduling_latency_s":1.701388,"redis_calls":10,"redis_duration_s":0.001242,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000201981,"duration_s":0.012078,"completed_at":"2026-11-10T09:52:20.345Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 151.765955] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.346Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"978ed94fec86f313d4603c41","created_at":"2026-11-10T09:52:18.644Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc2ffc02c9db74f7277e5a9fda197ac9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T09:52:18.653Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-978ed94fec86f313d4603c41: start","job_status":"start","queue_duration_s":1.693869,"scheduling_latency_s":1.693882}1423gitlab # [ 151.768881] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.353Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1424gitlab # [ 151.772114] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"978ed94fec86f313d4603c41","created_at":"2026-11-10T09:52:18.644Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc2ffc02c9db74f7277e5a9fda197ac9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T09:52:18.653Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-978ed94fec86f313d4603c41: done: 0.012262 sec","job_status":"done","queue_duration_s":1.693869,"scheduling_latency_s":1.693882,"redis_calls":10,"redis_duration_s":0.001256,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000198349,"duration_s":0.012262,"completed_at":"2026-11-10T09:52:20.359Z","load_balancing_strategy":"primary","db_duration_s":0.000296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 151.824806] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"7b2bd7a3d4419207029b602e","created_at":"2026-11-10T09:52:18.663Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"138113a33948a4cb0c90d586d52a221a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.765Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-7b2bd7a3d4419207029b602e: start","job_status":"start","queue_duration_s":1.594132,"scheduling_latency_s":1.594146}1426gitlab # [ 151.825661] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"7b2bd7a3d4419207029b602e","created_at":"2026-11-10T09:52:18.663Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"138113a33948a4cb0c90d586d52a221a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.765Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-7b2bd7a3d4419207029b602e: done: 0.015196 sec","job_status":"done","queue_duration_s":1.594132,"scheduling_latency_s":1.594146,"redis_calls":12,"redis_duration_s":0.001589,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000337194,"exclusive_lock_hold_duration_s":0.0035376,"duration_s":0.015196,"completed_at":"2026-11-10T09:52:20.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001698,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 151.880028] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9993a32c60ce9a5c842c35e5","created_at":"2026-11-10T09:52:18.769Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3409627b804ce757fee483044b6e1a0d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.804Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9993a32c60ce9a5c842c35e5: start","job_status":"start","queue_duration_s":1.571423,"scheduling_latency_s":1.571436}1428gitlab # [ 151.880888] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9993a32c60ce9a5c842c35e5","created_at":"2026-11-10T09:52:18.769Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3409627b804ce757fee483044b6e1a0d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.804Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9993a32c60ce9a5c842c35e5: done: 0.013349 sec","job_status":"done","queue_duration_s":1.571423,"scheduling_latency_s":1.571436,"redis_calls":12,"redis_duration_s":0.0016229999999999999,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000309257,"exclusive_lock_hold_duration_s":0.0020038869999999998,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013349,"completed_at":"2026-11-10T09:52:20.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 151.936760] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d427b7f3b942d7c231853751","created_at":"2026-11-10T09:52:18.813Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f02de78691f95e955412d7cbe58e9c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T09:52:18.830Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d427b7f3b942d7c231853751: start","job_status":"start","queue_duration_s":1.559867,"scheduling_latency_s":1.55988}1430gitlab # [ 151.937556] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.397Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1431gitlab # [ 151.938143] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d427b7f3b942d7c231853751","created_at":"2026-11-10T09:52:18.813Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f02de78691f95e955412d7cbe58e9c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T09:52:18.830Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d427b7f3b942d7c231853751: done: 0.011666 sec","job_status":"done","queue_duration_s":1.559867,"scheduling_latency_s":1.55988,"redis_calls":10,"redis_duration_s":0.001253,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.011666,"completed_at":"2026-11-10T09:52:20.402Z","load_balancing_strategy":"primary","db_duration_s":0.000253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 151.981764] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1c8431d9302c75f6cb42853f","created_at":"2026-11-10T09:52:18.841Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80049182053f63f8171543ea19dcad6a","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.848Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1c8431d9302c75f6cb42853f: start","job_status":"start","queue_duration_s":1.554486,"scheduling_latency_s":1.5545}1433gitlab # [ 151.982706] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1c8431d9302c75f6cb42853f","created_at":"2026-11-10T09:52:18.841Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80049182053f63f8171543ea19dcad6a","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.848Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1c8431d9302c75f6cb42853f: done: 0.017007 sec","job_status":"done","queue_duration_s":1.554486,"scheduling_latency_s":1.5545,"redis_calls":10,"redis_duration_s":0.001259,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000228242,"duration_s":0.017007,"completed_at":"2026-11-10T09:52:20.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 152.027255] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.421Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8106134f997fbbdf7871457b","created_at":"2026-11-10T09:52:18.922Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e2ee87581ffcdca3daf2bf6c3984704","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.930Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8106134f997fbbdf7871457b: start","job_status":"start","queue_duration_s":1.491099,"scheduling_latency_s":1.491113}1435gitlab # [ 152.028076] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.448Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8106134f997fbbdf7871457b","created_at":"2026-11-10T09:52:18.922Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e2ee87581ffcdca3daf2bf6c3984704","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.930Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8106134f997fbbdf7871457b: done: 0.027315 sec","job_status":"done","queue_duration_s":1.491099,"scheduling_latency_s":1.491113,"redis_calls":16,"redis_duration_s":0.002353,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001179,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024193,"duration_s":0.027315,"completed_at":"2026-11-10T09:52:20.448Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 152.066527] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ac0d94264f06bc468ca03fd2","created_at":"2026-11-10T09:52:18.945Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bdc113234889a78592d7a1696bf392a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.958Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ac0d94264f06bc468ca03fd2: start","job_status":"start","queue_duration_s":1.491313,"scheduling_latency_s":1.491328}1437gitlab # [ 152.109289] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.457Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1438gitlab # [ 152.110035] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.462Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ac0d94264f06bc468ca03fd2","created_at":"2026-11-10T09:52:18.945Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bdc113234889a78592d7a1696bf392a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.958Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ac0d94264f06bc468ca03fd2: done: 0.012948 sec","job_status":"done","queue_duration_s":1.491313,"scheduling_latency_s":1.491328,"redis_calls":10,"redis_duration_s":0.001302,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212038,"duration_s":0.012948,"completed_at":"2026-11-10T09:52:20.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000369,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 152.117286] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3fc58ec610f0dc10b84f216a","created_at":"2026-11-10T09:52:18.977Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6150b61380dc29ec7bdfde113a90c6f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.986Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3fc58ec610f0dc10b84f216a: start","job_status":"start","queue_duration_s":1.477201,"scheduling_latency_s":1.477214}1440gitlab # [ 152.164813] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.475Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3fc58ec610f0dc10b84f216a","created_at":"2026-11-10T09:52:18.977Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6150b61380dc29ec7bdfde113a90c6f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:18.986Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3fc58ec610f0dc10b84f216a: done: 0.012006 sec","job_status":"done","queue_duration_s":1.477201,"scheduling_latency_s":1.477214,"redis_calls":10,"redis_duration_s":0.001252,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022042,"duration_s":0.012006,"completed_at":"2026-11-10T09:52:20.475Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 152.167985] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"306a05e567c25784e57a6d33","created_at":"2026-11-10T09:52:18.995Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1682180eeaf37708063cedca6bf572bc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T09:52:19.004Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-306a05e567c25784e57a6d33: start","job_status":"start","queue_duration_s":1.47213,"scheduling_latency_s":1.472143}1442gitlab # [ 152.180172] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"306a05e567c25784e57a6d33","created_at":"2026-11-10T09:52:18.995Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1682180eeaf37708063cedca6bf572bc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T09:52:19.004Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-306a05e567c25784e57a6d33: done: 0.020843 sec","job_status":"done","queue_duration_s":1.47213,"scheduling_latency_s":1.472143,"redis_calls":10,"redis_duration_s":0.001345,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000242209,"duration_s":0.020843,"completed_at":"2026-11-10T09:52:20.497Z","load_balancing_strategy":"primary","db_duration_s":0.003208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 152.224785] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8b6248f315704eb915e7182a","created_at":"2026-11-10T09:52:19.018Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.029Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8b6248f315704eb915e7182a: start","job_status":"start","queue_duration_s":1.468911,"scheduling_latency_s":1.468925}1444gitlab # [ 152.228259] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8b6248f315704eb915e7182a","created_at":"2026-11-10T09:52:19.018Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.029Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8b6248f315704eb915e7182a: done: 0.023078 sec","job_status":"done","queue_duration_s":1.468911,"scheduling_latency_s":1.468925,"redis_calls":19,"redis_duration_s":0.002468,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00056,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000197,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001281,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227962,"duration_s":0.023078,"completed_at":"2026-11-10T09:52:20.521Z","load_balancing_strategy":"primary","db_duration_s":0.001497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 152.269709] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9014923ed262b74f4679a82e","created_at":"2026-11-10T09:52:19.043Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fd1231525a8bbb61a4e74be300d2f69","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.051Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9014923ed262b74f4679a82e: start","job_status":"start","queue_duration_s":1.470883,"scheduling_latency_s":1.470897}1446gitlab # [ 152.309837] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"9014923ed262b74f4679a82e","created_at":"2026-11-10T09:52:19.043Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fd1231525a8bbb61a4e74be300d2f69","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.051Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-9014923ed262b74f4679a82e: done: 0.023773 sec","job_status":"done","queue_duration_s":1.470883,"scheduling_latency_s":1.470897,"redis_calls":12,"redis_duration_s":0.001693,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000390553,"exclusive_lock_hold_duration_s":0.008073651,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.023773,"completed_at":"2026-11-10T09:52:20.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 152.355089] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6c8e3651236168f28ac5ed14","created_at":"2026-11-10T09:52:19.064Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a31325e26ed3d5a9ac56929e2b0fafc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.072Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6c8e3651236168f28ac5ed14: start","job_status":"start","queue_duration_s":1.474944,"scheduling_latency_s":1.474958}1448gitlab # [ 152.355772] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.554Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1449gitlab # [ 152.356479] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6c8e3651236168f28ac5ed14","created_at":"2026-11-10T09:52:19.064Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a31325e26ed3d5a9ac56929e2b0fafc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.072Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6c8e3651236168f28ac5ed14: done: 0.011392 sec","job_status":"done","queue_duration_s":1.474944,"scheduling_latency_s":1.474958,"redis_calls":10,"redis_duration_s":0.001333,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000206451,"duration_s":0.011392,"completed_at":"2026-11-10T09:52:20.558Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000325,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 152.407984] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9f82836d61f0d09b3ad28267","created_at":"2026-11-10T09:52:19.083Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"233839b79ace8cabdbc02f6904940bd5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.091Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9f82836d61f0d09b3ad28267: start","job_status":"start","queue_duration_s":1.467911,"scheduling_latency_s":1.467925}1451gitlab # [ 152.408765] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9f82836d61f0d09b3ad28267","created_at":"2026-11-10T09:52:19.083Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"233839b79ace8cabdbc02f6904940bd5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T09:52:19.091Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9f82836d61f0d09b3ad28267: done: 0.015312 sec","job_status":"done","queue_duration_s":1.467911,"scheduling_latency_s":1.467925,"redis_calls":10,"redis_duration_s":0.001295,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208686,"duration_s":0.015312,"completed_at":"2026-11-10T09:52:20.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 152.455244] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.575Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e66cc61f4b2db0721f47671d","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e66cc61f4b2db0721f47671d: start","job_status":"start","queue_duration_s":0.060412,"scheduling_latency_s":0.060426}1453gitlab # [ 152.456128] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.586Z","message":"No existing merge request to be cleaned up."}1454gitlab # [ 152.457145] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.594Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e66cc61f4b2db0721f47671d","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e66cc61f4b2db0721f47671d: done: 0.018627 sec","job_status":"done","queue_duration_s":0.060412,"scheduling_latency_s":0.060426,"redis_calls":13,"redis_duration_s":0.001826,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025087,"duration_s":0.018627,"completed_at":"2026-11-10T09:52:20.594Z","load_balancing_strategy":"primary","db_duration_s":0.002538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 152.497417] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.595Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bdcd465553f9346e8dfc67d5","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bdcd465553f9346e8dfc67d5: start","job_status":"start","queue_duration_s":0.079937,"scheduling_latency_s":0.079952}1456gitlab # [ 152.498302] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.602Z","message":"No existing merge request to be cleaned up."}1457gitlab # [ 152.499077] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.607Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bdcd465553f9346e8dfc67d5","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bdcd465553f9346e8dfc67d5: done: 0.012278 sec","job_status":"done","queue_duration_s":0.079937,"scheduling_latency_s":0.079952,"redis_calls":10,"redis_duration_s":0.001299,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012278,"completed_at":"2026-11-10T09:52:20.607Z","load_balancing_strategy":"primary","db_duration_s":0.001176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 152.541704] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.608Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"afba88022202421d0cb40578","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-afba88022202421d0cb40578: start","job_status":"start","queue_duration_s":0.093062,"scheduling_latency_s":0.093084}1459gitlab # [ 152.542662] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.615Z","message":"No existing merge request to be cleaned up."}1460gitlab # [ 152.543627] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.619Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"afba88022202421d0cb40578","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-afba88022202421d0cb40578: done: 0.011186 sec","job_status":"done","queue_duration_s":0.093062,"scheduling_latency_s":0.093084,"redis_calls":10,"redis_duration_s":0.0013419999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000813,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011186,"completed_at":"2026-11-10T09:52:20.619Z","load_balancing_strategy":"primary","db_duration_s":0.001034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 152.590459] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.620Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"52869a6de311f7117bd07d35","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-52869a6de311f7117bd07d35: start","job_status":"start","queue_duration_s":0.105095,"scheduling_latency_s":0.10511}1462gitlab # [ 152.591197] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.627Z","message":"No existing merge request to be cleaned up."}1463gitlab # [ 152.591617] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:52:20.631Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"52869a6de311f7117bd07d35","created_at":"2026-11-10T09:52:20.508Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-6e793635ff3e4f96","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"abf0824b7036e65a75ffb6f96cc273df","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:52:20.515Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-52869a6de311f7117bd07d35: done: 0.011293 sec","job_status":"done","queue_duration_s":0.105095,"scheduling_latency_s":0.10511,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011293,"completed_at":"2026-11-10T09:52:20.631Z","load_balancing_strategy":"primary","db_duration_s":0.000984,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 152.806115] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2389993589 wd_nsec: 23899924261465gitlab # [ 169.439357] setup-gitlab-start[2280]: W, [2026-11-10T09:52:39.663424 #2280] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1466gitlab # [ 191.171470] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a2f02145fd5bb369b2438947","created_at":"2026-11-10T09:53:01.357Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd199010ba7f6eb29fe32311b65dc944","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.391Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a2f02145fd5bb369b2438947: start","job_status":"start","queue_duration_s":0.002248,"scheduling_latency_s":0.002298}1467gitlab # [ 191.250338] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a2f02145fd5bb369b2438947","created_at":"2026-11-10T09:53:01.357Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd199010ba7f6eb29fe32311b65dc944","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.391Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a2f02145fd5bb369b2438947: done: 0.080505 sec","job_status":"done","queue_duration_s":0.002248,"scheduling_latency_s":0.002298,"redis_calls":7,"redis_duration_s":0.010043000000000002,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001659,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080505,"completed_at":"2026-11-10T09:53:01.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 191.256976] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"793335c4a00092b10b2fcc59","created_at":"2026-11-10T09:53:01.420Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e1009da576e6c78c5c2a96e8fb19148","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.425Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-793335c4a00092b10b2fcc59: start","job_status":"start","queue_duration_s":0.05616,"scheduling_latency_s":0.056206}1469gitlab # [ 191.382069] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"793335c4a00092b10b2fcc59","created_at":"2026-11-10T09:53:01.420Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e1009da576e6c78c5c2a96e8fb19148","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.425Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-793335c4a00092b10b2fcc59: done: 0.125177 sec","job_status":"done","queue_duration_s":0.05616,"scheduling_latency_s":0.056206,"redis_calls":9,"redis_duration_s":0.012919,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009069,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001191772,"exclusive_lock_hold_duration_s":0.07771742,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.125177,"completed_at":"2026-11-10T09:53:01.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.088917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 191.421031] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e24e339d61ef4c889ab39231","created_at":"2026-11-10T09:53:01.440Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f42fbf1180c210b5f0d028bb2746bdd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.447Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e24e339d61ef4c889ab39231: start","job_status":"start","queue_duration_s":0.163681,"scheduling_latency_s":0.163711}1471gitlab # [ 191.434071] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.619Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1472gitlab # [ 191.437705] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.631Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e24e339d61ef4c889ab39231","created_at":"2026-11-10T09:53:01.440Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f42fbf1180c210b5f0d028bb2746bdd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.447Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e24e339d61ef4c889ab39231: done: 0.021024 sec","job_status":"done","queue_duration_s":0.163681,"scheduling_latency_s":0.163711,"redis_calls":7,"redis_duration_s":0.003334,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021024,"completed_at":"2026-11-10T09:53:01.631Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 191.474241] postgres[2474]: [2474] ERROR: relation "table_without_models" does not exist1474gitlab # [ 191.474867] postgres[2474]: [2474] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1475gitlab # [ 191.475497] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1f825accd134a429380e4995","created_at":"2026-11-10T09:53:01.471Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee89f97cefffc2a126c8ba093903ae7f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.479Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1f825accd134a429380e4995: start","job_status":"start","queue_duration_s":0.153322,"scheduling_latency_s":0.153369}1476gitlab # [ 191.504087] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1f825accd134a429380e4995","created_at":"2026-11-10T09:53:01.471Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee89f97cefffc2a126c8ba093903ae7f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.479Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1f825accd134a429380e4995: done: 0.095736 sec","job_status":"done","queue_duration_s":0.153322,"scheduling_latency_s":0.153369,"redis_calls":7,"redis_duration_s":0.013104999999999999,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011508,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001597,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.095736,"completed_at":"2026-11-10T09:53:01.728Z","load_balancing_strategy":"primary","db_duration_s":0.094683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 191.505197] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"950c6014f1ebd31e2b8286af","created_at":"2026-11-10T09:53:01.497Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.500Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-950c6014f1ebd31e2b8286af: start","job_status":"start","queue_duration_s":0.229605,"scheduling_latency_s":0.229624}1478gitlab # [ 191.564631] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"950c6014f1ebd31e2b8286af","created_at":"2026-11-10T09:53:01.497Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.500Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-950c6014f1ebd31e2b8286af: done: 0.059369 sec","job_status":"done","queue_duration_s":0.229605,"scheduling_latency_s":0.229624,"redis_calls":60,"redis_duration_s":0.012213,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000545,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.011271,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059369,"completed_at":"2026-11-10T09:53:01.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 191.602907] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.827Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"2fd57516b36f0397f5ed4b0d","created_at":"2026-11-10T09:53:01.510Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe20bbbca0c4a7be8a513fd29aa21099","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.520Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-2fd57516b36f0397f5ed4b0d: start","job_status":"start","queue_duration_s":0.307176,"scheduling_latency_s":0.307214}1480gitlab # [ 191.641980] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"2fd57516b36f0397f5ed4b0d","created_at":"2026-11-10T09:53:01.510Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe20bbbca0c4a7be8a513fd29aa21099","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.520Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-2fd57516b36f0397f5ed4b0d: done: 0.039282 sec","job_status":"done","queue_duration_s":0.307176,"scheduling_latency_s":0.307214,"redis_calls":20,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":20,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001321,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000223,"redis_queues_read_bytes":3,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227962,"duration_s":0.039282,"completed_at":"2026-11-10T09:53:01.866Z","load_balancing_strategy":"primary","db_duration_s":0.003072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 191.681577] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3e4a8ea0495691f2fd5201ed","created_at":"2026-11-10T09:53:01.532Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc722e30416c8dc2020373ea841291a5","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.537Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3e4a8ea0495691f2fd5201ed: start","job_status":"start","queue_duration_s":0.36833,"scheduling_latency_s":0.368355}1482gitlab # [ 191.727737] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3e4a8ea0495691f2fd5201ed","created_at":"2026-11-10T09:53:01.532Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc722e30416c8dc2020373ea841291a5","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.537Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3e4a8ea0495691f2fd5201ed: done: 0.04588 sec","job_status":"done","queue_duration_s":0.36833,"scheduling_latency_s":0.368355,"redis_calls":11,"redis_duration_s":0.0023740000000000002,"redis_read_bytes":136,"redis_write_bytes":1288,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000224,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007549283,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04588,"completed_at":"2026-11-10T09:53:01.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003603,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1483gitlab # [ 191.769220] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:01.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"08e57f5c158f5b2d1a4759b9","created_at":"2026-11-10T09:53:01.556Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9030145df146a288f3eb8b0523e7a69","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.559Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-08e57f5c158f5b2d1a4759b9: start","job_status":"start","queue_duration_s":0.434536,"scheduling_latency_s":0.434571}1484gitlab # [ 192.013630] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"08e57f5c158f5b2d1a4759b9","created_at":"2026-11-10T09:53:01.556Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9030145df146a288f3eb8b0523e7a69","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.559Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-08e57f5c158f5b2d1a4759b9: done: 0.244415 sec","job_status":"done","queue_duration_s":0.434536,"scheduling_latency_s":0.434571,"redis_calls":562,"redis_duration_s":0.07401,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.073117,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.183876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.244415,"completed_at":"2026-11-10T09:53:02.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 192.016223] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.239Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a22955630c121f098dac3d74","created_at":"2026-11-10T09:53:01.569Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af5c5df41be2caeb248a4e01422a731d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.575Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a22955630c121f098dac3d74: start","job_status":"start","queue_duration_s":0.663762,"scheduling_latency_s":0.663784}1486gitlab # [ 192.261326] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.485Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a22955630c121f098dac3d74","created_at":"2026-11-10T09:53:01.569Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af5c5df41be2caeb248a4e01422a731d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.575Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a22955630c121f098dac3d74: done: 0.245887 sec","job_status":"done","queue_duration_s":0.663762,"scheduling_latency_s":0.663784,"redis_calls":9,"redis_duration_s":0.0020410000000000003,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000339708,"exclusive_lock_hold_duration_s":0.234394316,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.245887,"completed_at":"2026-11-10T09:53:02.485Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.131625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 192.264344] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cffbb1c4c347e79597b762c1","created_at":"2026-11-10T09:53:01.583Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"930e33322444ac573ba70e3fca7d4de6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.585Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cffbb1c4c347e79597b762c1: start","job_status":"start","queue_duration_s":0.901099,"scheduling_latency_s":0.901115}1488gitlab # [ 192.287200] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cffbb1c4c347e79597b762c1","created_at":"2026-11-10T09:53:01.583Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"930e33322444ac573ba70e3fca7d4de6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.585Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cffbb1c4c347e79597b762c1: done: 0.011811 sec","job_status":"done","queue_duration_s":0.901099,"scheduling_latency_s":0.901115,"redis_calls":9,"redis_duration_s":0.001525,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000631,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000322387,"exclusive_lock_hold_duration_s":0.002641118,"duration_s":0.011811,"completed_at":"2026-11-10T09:53:02.498Z","load_balancing_strategy":"primary","db_duration_s":0.001092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 192.354403] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa76ffb78cbe8d218eb46e9c","created_at":"2026-11-10T09:53:01.599Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e94c5c760fbd8c55dc811dea677781da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.602Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa76ffb78cbe8d218eb46e9c: start","job_status":"start","queue_duration_s":0.896629,"scheduling_latency_s":0.896642}1490gitlab # [ 192.358909] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.503Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1491gitlab # [ 192.361946] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa76ffb78cbe8d218eb46e9c","created_at":"2026-11-10T09:53:01.599Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e94c5c760fbd8c55dc811dea677781da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.602Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa76ffb78cbe8d218eb46e9c: done: 0.007826 sec","job_status":"done","queue_duration_s":0.896629,"scheduling_latency_s":0.896642,"redis_calls":7,"redis_duration_s":0.000998,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007826,"completed_at":"2026-11-10T09:53:02.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 192.405029] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2d2636421e893910c20fe862","created_at":"2026-11-10T09:53:01.608Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad7e15027639f2bc6b42e85cb74625fd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T09:53:01.610Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2d2636421e893910c20fe862: start","job_status":"start","queue_duration_s":0.897359,"scheduling_latency_s":0.897373}1493gitlab # [ 192.410091] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.512Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1494gitlab # [ 192.413257] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.515Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2d2636421e893910c20fe862","created_at":"2026-11-10T09:53:01.608Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad7e15027639f2bc6b42e85cb74625fd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T09:53:01.610Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2d2636421e893910c20fe862: done: 0.00758 sec","job_status":"done","queue_duration_s":0.897359,"scheduling_latency_s":0.897373,"redis_calls":7,"redis_duration_s":0.000888,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00758,"completed_at":"2026-11-10T09:53:02.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 192.450300] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.516Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"57d761dd75a84a4dc881cf5c","created_at":"2026-11-10T09:53:01.624Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c0184b1cfd104b11a52be1a74b1426d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.628Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-57d761dd75a84a4dc881cf5c: start","job_status":"start","queue_duration_s":0.888161,"scheduling_latency_s":0.888174}1496gitlab # [ 192.488785] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"57d761dd75a84a4dc881cf5c","created_at":"2026-11-10T09:53:01.624Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c0184b1cfd104b11a52be1a74b1426d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.628Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-57d761dd75a84a4dc881cf5c: done: 0.010226 sec","job_status":"done","queue_duration_s":0.888161,"scheduling_latency_s":0.888174,"redis_calls":9,"redis_duration_s":0.001268,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000324622,"exclusive_lock_hold_duration_s":0.001942146,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.010226,"completed_at":"2026-11-10T09:53:02.526Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 192.533291] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fb1033502a88c9b025ffe202","created_at":"2026-11-10T09:53:01.638Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"239040d72646c32bf589663d7d471f67","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T09:53:01.640Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fb1033502a88c9b025ffe202: start","job_status":"start","queue_duration_s":0.887199,"scheduling_latency_s":0.887211}1498gitlab # [ 192.534051] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.531Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1499gitlab # [ 192.534750] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fb1033502a88c9b025ffe202","created_at":"2026-11-10T09:53:01.638Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"239040d72646c32bf589663d7d471f67","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T09:53:01.640Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fb1033502a88c9b025ffe202: done: 0.007447 sec","job_status":"done","queue_duration_s":0.887199,"scheduling_latency_s":0.887211,"redis_calls":7,"redis_duration_s":0.000808,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000226,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007447,"completed_at":"2026-11-10T09:53:02.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 192.584236] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.535Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"16d2dc9877b0c2c7f2364cea","created_at":"2026-11-10T09:53:01.655Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5eed2e984c2b15de6f3a911377caa3a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.657Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-16d2dc9877b0c2c7f2364cea: start","job_status":"start","queue_duration_s":0.877921,"scheduling_latency_s":0.877934}1501gitlab # [ 192.584956] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.615Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"16d2dc9877b0c2c7f2364cea","created_at":"2026-11-10T09:53:01.655Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5eed2e984c2b15de6f3a911377caa3a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.657Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-16d2dc9877b0c2c7f2364cea: done: 0.079882 sec","job_status":"done","queue_duration_s":0.877921,"scheduling_latency_s":0.877934,"redis_calls":7,"redis_duration_s":0.001072,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.068,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.079882,"completed_at":"2026-11-10T09:53:02.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.067882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 192.625265] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0111c086029cd500cc971758","created_at":"2026-11-10T09:53:01.664Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1316fb8eb526542f31aed0c6e6936db4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.666Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0111c086029cd500cc971758: start","job_status":"start","queue_duration_s":0.950517,"scheduling_latency_s":0.950539}1503gitlab # [ 192.625862] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.621Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1504gitlab # [ 192.626529] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0111c086029cd500cc971758","created_at":"2026-11-10T09:53:01.664Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1316fb8eb526542f31aed0c6e6936db4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.666Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0111c086029cd500cc971758: done: 0.008516 sec","job_status":"done","queue_duration_s":0.950517,"scheduling_latency_s":0.950539,"redis_calls":7,"redis_duration_s":0.001001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008516,"completed_at":"2026-11-10T09:53:02.625Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 192.674693] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1aa985c27706c935f6535808","created_at":"2026-11-10T09:53:01.675Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ead7737f2c0d1a31707777ff86b8ca99","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.677Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1aa985c27706c935f6535808: start","job_status":"start","queue_duration_s":0.948099,"scheduling_latency_s":0.948112}1506gitlab # [ 192.675540] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.634Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1aa985c27706c935f6535808","created_at":"2026-11-10T09:53:01.675Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ead7737f2c0d1a31707777ff86b8ca99","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.677Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1aa985c27706c935f6535808: done: 0.008281 sec","job_status":"done","queue_duration_s":0.948099,"scheduling_latency_s":0.948112,"redis_calls":8,"redis_duration_s":0.001003,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00013,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008281,"completed_at":"2026-11-10T09:53:02.634Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 192.724673] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3dc40dcb2de0354495671992","created_at":"2026-11-10T09:53:01.679Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e697833526e4cb986f038b1571f84f0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T09:53:01.680Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3dc40dcb2de0354495671992: start","job_status":"start","queue_duration_s":0.954275,"scheduling_latency_s":0.954288}1508gitlab # [ 192.725405] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3dc40dcb2de0354495671992","created_at":"2026-11-10T09:53:01.679Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e697833526e4cb986f038b1571f84f0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T09:53:01.680Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3dc40dcb2de0354495671992: done: 0.013497 sec","job_status":"done","queue_duration_s":0.954275,"scheduling_latency_s":0.954288,"redis_calls":7,"redis_duration_s":0.0009630000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013497,"completed_at":"2026-11-10T09:53:02.648Z","load_balancing_strategy":"primary","db_duration_s":0.00134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 192.774078] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b80d9c2bdbf7b749ba0c52f8","created_at":"2026-11-10T09:53:01.684Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.686Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b80d9c2bdbf7b749ba0c52f8: start","job_status":"start","queue_duration_s":0.963163,"scheduling_latency_s":0.963179}1510gitlab # [ 192.774745] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b80d9c2bdbf7b749ba0c52f8","created_at":"2026-11-10T09:53:01.684Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.686Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b80d9c2bdbf7b749ba0c52f8: done: 0.015391 sec","job_status":"done","queue_duration_s":0.963163,"scheduling_latency_s":0.963179,"redis_calls":14,"redis_duration_s":0.001887,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000207,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015391,"completed_at":"2026-11-10T09:53:02.664Z","load_balancing_strategy":"primary","db_duration_s":0.00082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 192.820598] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"c6648a46e0d3c8c57c8ea4fa","created_at":"2026-11-10T09:53:01.688Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28296592db6011d874018a7f649a92d9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.696Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-c6648a46e0d3c8c57c8ea4fa: start","job_status":"start","queue_duration_s":0.969083,"scheduling_latency_s":0.969099}1512gitlab # [ 192.821547] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"c6648a46e0d3c8c57c8ea4fa","created_at":"2026-11-10T09:53:01.688Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28296592db6011d874018a7f649a92d9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.696Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-c6648a46e0d3c8c57c8ea4fa: done: 0.019361 sec","job_status":"done","queue_duration_s":0.969083,"scheduling_latency_s":0.969099,"redis_calls":12,"redis_duration_s":0.00163,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000342781,"exclusive_lock_hold_duration_s":0.007699862,"duration_s":0.019361,"completed_at":"2026-11-10T09:53:02.685Z","load_balancing_strategy":"primary","db_duration_s":0.003598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 192.868596] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7cf740f781c11d129d2e6381","created_at":"2026-11-10T09:53:01.699Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51554e96693fa512c4385df2c0972ee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.700Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7cf740f781c11d129d2e6381: start","job_status":"start","queue_duration_s":0.985129,"scheduling_latency_s":0.985145}1514gitlab # [ 192.869434] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7cf740f781c11d129d2e6381","created_at":"2026-11-10T09:53:01.699Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51554e96693fa512c4385df2c0972ee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.700Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7cf740f781c11d129d2e6381: done: 0.020959 sec","job_status":"done","queue_duration_s":0.985129,"scheduling_latency_s":0.985145,"redis_calls":9,"redis_duration_s":0.001206,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000355353,"exclusive_lock_hold_duration_s":0.008284572,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.020959,"completed_at":"2026-11-10T09:53:02.706Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 192.910552] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.707Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0e576c0e265bfda63f3878b7","created_at":"2026-11-10T09:53:01.705Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d175a5ab08b3e3bcfb812f8bf18a3f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.706Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0e576c0e265bfda63f3878b7: start","job_status":"start","queue_duration_s":1.00113,"scheduling_latency_s":1.001143}1516gitlab # [ 192.911190] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.712Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1517gitlab # [ 192.911853] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0e576c0e265bfda63f3878b7","created_at":"2026-11-10T09:53:01.705Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d175a5ab08b3e3bcfb812f8bf18a3f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.706Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0e576c0e265bfda63f3878b7: done: 0.007794 sec","job_status":"done","queue_duration_s":1.00113,"scheduling_latency_s":1.001143,"redis_calls":7,"redis_duration_s":0.0009010000000000001,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007794,"completed_at":"2026-11-10T09:53:02.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518gitlab # [ 192.966720] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"889915e3a0247fbb086e50ba","created_at":"2026-11-10T09:53:01.710Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9b702749b30740717730111a37e8432","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.712Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-889915e3a0247fbb086e50ba: start","job_status":"start","queue_duration_s":1.004392,"scheduling_latency_s":1.004405}1519gitlab # [ 192.967588] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"889915e3a0247fbb086e50ba","created_at":"2026-11-10T09:53:01.710Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-f578edc546674f5a","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9b702749b30740717730111a37e8432","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.712Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-889915e3a0247fbb086e50ba: done: 0.008934 sec","job_status":"done","queue_duration_s":1.004392,"scheduling_latency_s":1.004405,"redis_calls":7,"redis_duration_s":0.000912,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008934,"completed_at":"2026-11-10T09:53:02.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 193.008751] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffde399225a8a11071406926","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffde399225a8a11071406926: start","job_status":"start","queue_duration_s":0.942421,"scheduling_latency_s":0.942435}1521gitlab # [ 193.009595] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffde399225a8a11071406926","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffde399225a8a11071406926: done: 0.009365 sec","job_status":"done","queue_duration_s":0.942421,"scheduling_latency_s":0.942435,"redis_calls":13,"redis_duration_s":0.0014500000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000461,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009365,"completed_at":"2026-11-10T09:53:02.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 193.056725] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6202832df87fb5542fefb761","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6202832df87fb5542fefb761: start","job_status":"start","queue_duration_s":0.952603,"scheduling_latency_s":0.952617}1523gitlab # [ 193.057603] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6202832df87fb5542fefb761","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6202832df87fb5542fefb761: done: 0.008883 sec","job_status":"done","queue_duration_s":0.952603,"scheduling_latency_s":0.952617,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008883,"completed_at":"2026-11-10T09:53:02.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 193.111576] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d62feb7f7d5c8c8ab5b9d7bf","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d62feb7f7d5c8c8ab5b9d7bf: start","job_status":"start","queue_duration_s":0.962296,"scheduling_latency_s":0.962308}1525gitlab # [ 193.112521] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d62feb7f7d5c8c8ab5b9d7bf","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d62feb7f7d5c8c8ab5b9d7bf: done: 0.008835 sec","job_status":"done","queue_duration_s":0.962296,"scheduling_latency_s":0.962308,"redis_calls":13,"redis_duration_s":0.0013570000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008835,"completed_at":"2026-11-10T09:53:02.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 193.125367] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73f8a72cea605a372374e867","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73f8a72cea605a372374e867: start","job_status":"start","queue_duration_s":0.971979,"scheduling_latency_s":0.971992}1527gitlab # [ 193.173185] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73f8a72cea605a372374e867","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73f8a72cea605a372374e867: done: 0.009168 sec","job_status":"done","queue_duration_s":0.971979,"scheduling_latency_s":0.971992,"redis_calls":13,"redis_duration_s":0.0016090000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009168,"completed_at":"2026-11-10T09:53:02.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 193.174173] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5132925d11507164bcf88f4","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5132925d11507164bcf88f4: start","job_status":"start","queue_duration_s":0.982024,"scheduling_latency_s":0.982038}1529gitlab # [ 193.222199] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5132925d11507164bcf88f4","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5132925d11507164bcf88f4: done: 0.009144 sec","job_status":"done","queue_duration_s":0.982024,"scheduling_latency_s":0.982038,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009144,"completed_at":"2026-11-10T09:53:02.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 193.223194] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebe11abecc876ea470f4684","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebe11abecc876ea470f4684: start","job_status":"start","queue_duration_s":0.991971,"scheduling_latency_s":0.991984}1531gitlab # [ 193.266582] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebe11abecc876ea470f4684","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebe11abecc876ea470f4684: done: 0.008877 sec","job_status":"done","queue_duration_s":0.991971,"scheduling_latency_s":0.991984,"redis_calls":13,"redis_duration_s":0.001376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008877,"completed_at":"2026-11-10T09:53:02.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 193.267889] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"329f41115da608cbd1d363f8","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-329f41115da608cbd1d363f8: start","job_status":"start","queue_duration_s":1.001659,"scheduling_latency_s":1.001674}1533gitlab # [ 193.308915] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"329f41115da608cbd1d363f8","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-329f41115da608cbd1d363f8: done: 0.008871 sec","job_status":"done","queue_duration_s":1.001659,"scheduling_latency_s":1.001674,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008871,"completed_at":"2026-11-10T09:53:02.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 193.351994] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32f3821f25a5694c6a2601d6","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32f3821f25a5694c6a2601d6: start","job_status":"start","queue_duration_s":1.011337,"scheduling_latency_s":1.011351}1535gitlab # [ 193.352817] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32f3821f25a5694c6a2601d6","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32f3821f25a5694c6a2601d6: done: 0.008783 sec","job_status":"done","queue_duration_s":1.011337,"scheduling_latency_s":1.011351,"redis_calls":13,"redis_duration_s":0.001348,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008783,"completed_at":"2026-11-10T09:53:02.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 193.397171] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"329fceac9fda9e6f406c8cd5","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-329fceac9fda9e6f406c8cd5: start","job_status":"start","queue_duration_s":1.020936,"scheduling_latency_s":1.02095}1537gitlab # [ 193.398063] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"329fceac9fda9e6f406c8cd5","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-329fceac9fda9e6f406c8cd5: done: 0.009012 sec","job_status":"done","queue_duration_s":1.020936,"scheduling_latency_s":1.02095,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009012,"completed_at":"2026-11-10T09:53:02.813Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538gitlab # [ 193.447077] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1961e2fde0ee4efafd50753d","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1961e2fde0ee4efafd50753d: start","job_status":"start","queue_duration_s":1.030787,"scheduling_latency_s":1.0308}1539gitlab # [ 193.447810] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1961e2fde0ee4efafd50753d","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1961e2fde0ee4efafd50753d: done: 0.008753 sec","job_status":"done","queue_duration_s":1.030787,"scheduling_latency_s":1.0308,"redis_calls":13,"redis_duration_s":0.001368,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008753,"completed_at":"2026-11-10T09:53:02.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 193.496153] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f73b2ae43ee6ff8df3bf681","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f73b2ae43ee6ff8df3bf681: start","job_status":"start","queue_duration_s":1.040344,"scheduling_latency_s":1.040357}1541gitlab # [ 193.496912] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f73b2ae43ee6ff8df3bf681","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f73b2ae43ee6ff8df3bf681: done: 0.008794 sec","job_status":"done","queue_duration_s":1.040344,"scheduling_latency_s":1.040357,"redis_calls":13,"redis_duration_s":0.001352,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008794,"completed_at":"2026-11-10T09:53:02.833Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542gitlab # [ 193.541989] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b51aa79e9016c60af283331a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b51aa79e9016c60af283331a: start","job_status":"start","queue_duration_s":1.050016,"scheduling_latency_s":1.05003}1543gitlab # [ 193.542752] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b51aa79e9016c60af283331a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b51aa79e9016c60af283331a: done: 0.008781 sec","job_status":"done","queue_duration_s":1.050016,"scheduling_latency_s":1.05003,"redis_calls":13,"redis_duration_s":0.001353,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000443,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008781,"completed_at":"2026-11-10T09:53:02.842Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544gitlab # [ 193.588304] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"725cef6c736b49a3ebcea51f","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-725cef6c736b49a3ebcea51f: start","job_status":"start","queue_duration_s":1.059599,"scheduling_latency_s":1.059612}1545gitlab # [ 193.589076] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"725cef6c736b49a3ebcea51f","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-725cef6c736b49a3ebcea51f: done: 0.008946 sec","job_status":"done","queue_duration_s":1.059599,"scheduling_latency_s":1.059612,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008946,"completed_at":"2026-11-10T09:53:02.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 193.634473] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59b68eeba52af61a997b7de2","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59b68eeba52af61a997b7de2: start","job_status":"start","queue_duration_s":1.069407,"scheduling_latency_s":1.069422}1547gitlab # [ 193.635392] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59b68eeba52af61a997b7de2","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59b68eeba52af61a997b7de2: done: 0.009189 sec","job_status":"done","queue_duration_s":1.069407,"scheduling_latency_s":1.069422,"redis_calls":13,"redis_duration_s":0.001636,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009189,"completed_at":"2026-11-10T09:53:02.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548gitlab # [ 193.649819] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1d10cebae70fb37757843ea","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1d10cebae70fb37757843ea: start","job_status":"start","queue_duration_s":1.079416,"scheduling_latency_s":1.079429}1549gitlab # [ 193.662437] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1d10cebae70fb37757843ea","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1d10cebae70fb37757843ea: done: 0.008824 sec","job_status":"done","queue_duration_s":1.079416,"scheduling_latency_s":1.079429,"redis_calls":13,"redis_duration_s":0.001376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008824,"completed_at":"2026-11-10T09:53:02.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550gitlab # [ 193.709803] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5806fecf508bd0f4f2f98603","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5806fecf508bd0f4f2f98603: start","job_status":"start","queue_duration_s":1.089022,"scheduling_latency_s":1.089036}1551gitlab # [ 193.710756] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5806fecf508bd0f4f2f98603","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5806fecf508bd0f4f2f98603: done: 0.008821 sec","job_status":"done","queue_duration_s":1.089022,"scheduling_latency_s":1.089036,"redis_calls":13,"redis_duration_s":0.001369,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008821,"completed_at":"2026-11-10T09:53:02.881Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552gitlab # [ 193.748622] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88130372790cf19f45626f9","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88130372790cf19f45626f9: start","job_status":"start","queue_duration_s":1.098631,"scheduling_latency_s":1.098644}1553gitlab # [ 193.797179] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88130372790cf19f45626f9","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88130372790cf19f45626f9: done: 0.008721 sec","job_status":"done","queue_duration_s":1.098631,"scheduling_latency_s":1.098644,"redis_calls":13,"redis_duration_s":0.001346,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000427,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008721,"completed_at":"2026-11-10T09:53:02.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554gitlab # [ 193.798189] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60454827d9a19df02015c115","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60454827d9a19df02015c115: start","job_status":"start","queue_duration_s":1.108122,"scheduling_latency_s":1.108135}1555gitlab # [ 193.839093] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60454827d9a19df02015c115","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60454827d9a19df02015c115: done: 0.009198 sec","job_status":"done","queue_duration_s":1.108122,"scheduling_latency_s":1.108135,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009198,"completed_at":"2026-11-10T09:53:02.901Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556gitlab # [ 193.882064] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a64db5cdede65cc6d0b22159","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a64db5cdede65cc6d0b22159: start","job_status":"start","queue_duration_s":1.118206,"scheduling_latency_s":1.118221}1557gitlab # [ 193.882903] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a64db5cdede65cc6d0b22159","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a64db5cdede65cc6d0b22159: done: 0.009362 sec","job_status":"done","queue_duration_s":1.118206,"scheduling_latency_s":1.118221,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009362,"completed_at":"2026-11-10T09:53:02.911Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558gitlab # [ 193.891063] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35990380bd9011a19462108b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35990380bd9011a19462108b: start","job_status":"start","queue_duration_s":1.128504,"scheduling_latency_s":1.128517}1559gitlab # [ 193.942442] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35990380bd9011a19462108b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35990380bd9011a19462108b: done: 0.008872 sec","job_status":"done","queue_duration_s":1.128504,"scheduling_latency_s":1.128517,"redis_calls":13,"redis_duration_s":0.001387,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008872,"completed_at":"2026-11-10T09:53:02.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560gitlab # [ 193.943675] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43a0d1a3e637a2e732d6706a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43a0d1a3e637a2e732d6706a: start","job_status":"start","queue_duration_s":1.138145,"scheduling_latency_s":1.13816}1561gitlab # [ 193.984039] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43a0d1a3e637a2e732d6706a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43a0d1a3e637a2e732d6706a: done: 0.008739 sec","job_status":"done","queue_duration_s":1.138145,"scheduling_latency_s":1.13816,"redis_calls":13,"redis_duration_s":0.00134,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000435,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008739,"completed_at":"2026-11-10T09:53:02.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562gitlab # [ 193.986476] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75a7c249103d314fb45a63be","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75a7c249103d314fb45a63be: start","job_status":"start","queue_duration_s":1.147694,"scheduling_latency_s":1.147709}1563gitlab # [ 193.998183] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75a7c249103d314fb45a63be","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75a7c249103d314fb45a63be: done: 0.008821 sec","job_status":"done","queue_duration_s":1.147694,"scheduling_latency_s":1.147709,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008821,"completed_at":"2026-11-10T09:53:02.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564gitlab # [ 194.038656] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4db83ecd10ab6b8f454c4b5","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4db83ecd10ab6b8f454c4b5: start","job_status":"start","queue_duration_s":1.157323,"scheduling_latency_s":1.157337}1565gitlab # [ 194.045788] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4db83ecd10ab6b8f454c4b5","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4db83ecd10ab6b8f454c4b5: done: 0.008811 sec","job_status":"done","queue_duration_s":1.157323,"scheduling_latency_s":1.157337,"redis_calls":13,"redis_duration_s":0.001342,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000435,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008811,"completed_at":"2026-11-10T09:53:02.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566gitlab # [ 194.108081] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"030a5d957da6815e83315c03","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-030a5d957da6815e83315c03: start","job_status":"start","queue_duration_s":1.167072,"scheduling_latency_s":1.167085}1567gitlab # [ 194.108799] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"030a5d957da6815e83315c03","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-030a5d957da6815e83315c03: done: 0.00918 sec","job_status":"done","queue_duration_s":1.167072,"scheduling_latency_s":1.167085,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00918,"completed_at":"2026-11-10T09:53:02.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568gitlab # [ 194.154152] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c83d1cd734b5298bad50fd41","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c83d1cd734b5298bad50fd41: start","job_status":"start","queue_duration_s":1.177119,"scheduling_latency_s":1.177133}1569gitlab # [ 194.154893] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c83d1cd734b5298bad50fd41","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c83d1cd734b5298bad50fd41: done: 0.00881 sec","job_status":"done","queue_duration_s":1.177119,"scheduling_latency_s":1.177133,"redis_calls":13,"redis_duration_s":0.001377,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000435,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00881,"completed_at":"2026-11-10T09:53:02.969Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570gitlab # [ 194.203053] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b1653c8f58cc5a26a8d067d","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b1653c8f58cc5a26a8d067d: start","job_status":"start","queue_duration_s":1.186735,"scheduling_latency_s":1.186748}1571gitlab # [ 194.203848] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b1653c8f58cc5a26a8d067d","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b1653c8f58cc5a26a8d067d: done: 0.008776 sec","job_status":"done","queue_duration_s":1.186735,"scheduling_latency_s":1.186748,"redis_calls":13,"redis_duration_s":0.001389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008776,"completed_at":"2026-11-10T09:53:02.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572gitlab # [ 194.247241] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28dd8b3ce8e03419c7b84259","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28dd8b3ce8e03419c7b84259: start","job_status":"start","queue_duration_s":1.196331,"scheduling_latency_s":1.196344}1573gitlab # [ 194.248028] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28dd8b3ce8e03419c7b84259","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28dd8b3ce8e03419c7b84259: done: 0.008713 sec","job_status":"done","queue_duration_s":1.196331,"scheduling_latency_s":1.196344,"redis_calls":13,"redis_duration_s":0.00135,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008713,"completed_at":"2026-11-10T09:53:02.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574gitlab # [ 194.291680] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"753c6bc9e586f4fe6ea2ff93","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-753c6bc9e586f4fe6ea2ff93: start","job_status":"start","queue_duration_s":1.20584,"scheduling_latency_s":1.205854}1575gitlab # [ 194.292585] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:02.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"753c6bc9e586f4fe6ea2ff93","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-753c6bc9e586f4fe6ea2ff93: done: 0.008801 sec","job_status":"done","queue_duration_s":1.20584,"scheduling_latency_s":1.205854,"redis_calls":13,"redis_duration_s":0.0013830000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008801,"completed_at":"2026-11-10T09:53:02.998Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576gitlab # [ 194.343332] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22024b7f5b5200f72b654737","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22024b7f5b5200f72b654737: start","job_status":"start","queue_duration_s":1.218979,"scheduling_latency_s":1.218994}1577gitlab # [ 194.344082] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22024b7f5b5200f72b654737","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22024b7f5b5200f72b654737: done: 0.00928 sec","job_status":"done","queue_duration_s":1.218979,"scheduling_latency_s":1.218994,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00928,"completed_at":"2026-11-10T09:53:03.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578gitlab # [ 194.405862] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c02473809f2ada8810e28419","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c02473809f2ada8810e28419: start","job_status":"start","queue_duration_s":1.229049,"scheduling_latency_s":1.229062}1579gitlab # [ 194.406653] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c02473809f2ada8810e28419","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c02473809f2ada8810e28419: done: 0.00888 sec","job_status":"done","queue_duration_s":1.229049,"scheduling_latency_s":1.229062,"redis_calls":13,"redis_duration_s":0.0013549999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000437,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00888,"completed_at":"2026-11-10T09:53:03.021Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580gitlab # [ 194.446897] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9acae198b2c06635293a41d","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9acae198b2c06635293a41d: start","job_status":"start","queue_duration_s":1.238761,"scheduling_latency_s":1.238774}1581gitlab # [ 194.447825] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9acae198b2c06635293a41d","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9acae198b2c06635293a41d: done: 0.038807 sec","job_status":"done","queue_duration_s":1.238761,"scheduling_latency_s":1.238774,"redis_calls":13,"redis_duration_s":0.001716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038807,"completed_at":"2026-11-10T09:53:03.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582gitlab # [ 194.489774] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bf27d29ac2aeea100571ff2","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bf27d29ac2aeea100571ff2: start","job_status":"start","queue_duration_s":1.278561,"scheduling_latency_s":1.278577}1583gitlab # [ 194.490789] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bf27d29ac2aeea100571ff2","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bf27d29ac2aeea100571ff2: done: 0.009513 sec","job_status":"done","queue_duration_s":1.278561,"scheduling_latency_s":1.278577,"redis_calls":13,"redis_duration_s":0.0013779999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009513,"completed_at":"2026-11-10T09:53:03.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584gitlab # [ 194.532751] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75d7752d45c1c769d44c9b0a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75d7752d45c1c769d44c9b0a: start","job_status":"start","queue_duration_s":1.288912,"scheduling_latency_s":1.288927}1585gitlab # [ 194.533346] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75d7752d45c1c769d44c9b0a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75d7752d45c1c769d44c9b0a: done: 0.009897 sec","job_status":"done","queue_duration_s":1.288912,"scheduling_latency_s":1.288927,"redis_calls":13,"redis_duration_s":0.001379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009897,"completed_at":"2026-11-10T09:53:03.082Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586gitlab # [ 194.573608] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0da48f7fbb4fb781402a51b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0da48f7fbb4fb781402a51b: start","job_status":"start","queue_duration_s":1.299746,"scheduling_latency_s":1.29976}1587gitlab # [ 194.574292] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0da48f7fbb4fb781402a51b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0da48f7fbb4fb781402a51b: done: 0.012286 sec","job_status":"done","queue_duration_s":1.299746,"scheduling_latency_s":1.29976,"redis_calls":13,"redis_duration_s":0.0018310000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012286,"completed_at":"2026-11-10T09:53:03.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588gitlab # [ 194.625199] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7543d7fe0be8a5b533e1db25","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7543d7fe0be8a5b533e1db25: start","job_status":"start","queue_duration_s":1.313588,"scheduling_latency_s":1.31361}1589gitlab # [ 194.625991] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7543d7fe0be8a5b533e1db25","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7543d7fe0be8a5b533e1db25: done: 0.011398 sec","job_status":"done","queue_duration_s":1.313588,"scheduling_latency_s":1.31361,"redis_calls":13,"redis_duration_s":0.001791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011398,"completed_at":"2026-11-10T09:53:03.108Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590gitlab # [ 194.673204] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cc5e342d9c6c3771b7b13cc","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cc5e342d9c6c3771b7b13cc: start","job_status":"start","queue_duration_s":1.325814,"scheduling_latency_s":1.325827}1591gitlab # [ 194.673995] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cc5e342d9c6c3771b7b13cc","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cc5e342d9c6c3771b7b13cc: done: 0.00937 sec","job_status":"done","queue_duration_s":1.325814,"scheduling_latency_s":1.325827,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00937,"completed_at":"2026-11-10T09:53:03.119Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592gitlab # [ 194.719296] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b519167877e105e47e0c425a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b519167877e105e47e0c425a: start","job_status":"start","queue_duration_s":1.336496,"scheduling_latency_s":1.336509}1593gitlab # [ 194.720078] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b519167877e105e47e0c425a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b519167877e105e47e0c425a: done: 0.009426 sec","job_status":"done","queue_duration_s":1.336496,"scheduling_latency_s":1.336509,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009426,"completed_at":"2026-11-10T09:53:03.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594gitlab # [ 194.747780] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba4c8bd9316c6fd0e137e05","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba4c8bd9316c6fd0e137e05: start","job_status":"start","queue_duration_s":1.346761,"scheduling_latency_s":1.346774}1595gitlab # [ 194.787960] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba4c8bd9316c6fd0e137e05","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba4c8bd9316c6fd0e137e05: done: 0.009681 sec","job_status":"done","queue_duration_s":1.346761,"scheduling_latency_s":1.346774,"redis_calls":13,"redis_duration_s":0.00174,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009681,"completed_at":"2026-11-10T09:53:03.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596gitlab # [ 194.831911] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04eae3675a488ee4450d74f4","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04eae3675a488ee4450d74f4: start","job_status":"start","queue_duration_s":1.357325,"scheduling_latency_s":1.357338}1597gitlab # [ 194.832400] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04eae3675a488ee4450d74f4","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04eae3675a488ee4450d74f4: done: 0.009467 sec","job_status":"done","queue_duration_s":1.357325,"scheduling_latency_s":1.357338,"redis_calls":13,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009467,"completed_at":"2026-11-10T09:53:03.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598gitlab # [ 194.880883] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de5f955743045e352006792b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de5f955743045e352006792b: start","job_status":"start","queue_duration_s":1.367648,"scheduling_latency_s":1.367668}1599gitlab # [ 194.881773] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de5f955743045e352006792b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de5f955743045e352006792b: done: 0.010735 sec","job_status":"done","queue_duration_s":1.367648,"scheduling_latency_s":1.367668,"redis_calls":13,"redis_duration_s":0.001652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010735,"completed_at":"2026-11-10T09:53:03.162Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600gitlab # [ 194.889907] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a987227992df5791dc3cc578","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a987227992df5791dc3cc578: start","job_status":"start","queue_duration_s":1.379261,"scheduling_latency_s":1.379274}1601gitlab # [ 194.938997] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a987227992df5791dc3cc578","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a987227992df5791dc3cc578: done: 0.009595 sec","job_status":"done","queue_duration_s":1.379261,"scheduling_latency_s":1.379274,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009595,"completed_at":"2026-11-10T09:53:03.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602gitlab # [ 194.940091] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc8ddda94134d1a2f19c6259","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc8ddda94134d1a2f19c6259: start","job_status":"start","queue_duration_s":1.3897,"scheduling_latency_s":1.389713}1603gitlab # [ 194.949159] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc8ddda94134d1a2f19c6259","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc8ddda94134d1a2f19c6259: done: 0.009382 sec","job_status":"done","queue_duration_s":1.3897,"scheduling_latency_s":1.389713,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009382,"completed_at":"2026-11-10T09:53:03.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604gitlab # [ 194.993166] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c20afdedfe5cfe1c777bf9a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c20afdedfe5cfe1c777bf9a: start","job_status":"start","queue_duration_s":1.39993,"scheduling_latency_s":1.399943}1605gitlab # [ 194.996724] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c20afdedfe5cfe1c777bf9a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c20afdedfe5cfe1c777bf9a: done: 0.010188 sec","job_status":"done","queue_duration_s":1.39993,"scheduling_latency_s":1.399943,"redis_calls":13,"redis_duration_s":0.001628,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010188,"completed_at":"2026-11-10T09:53:03.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606gitlab # [ 195.042072] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b3214e7bf3026159746d814","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b3214e7bf3026159746d814: start","job_status":"start","queue_duration_s":1.411009,"scheduling_latency_s":1.411023}1607gitlab # [ 195.042780] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b3214e7bf3026159746d814","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b3214e7bf3026159746d814: done: 0.009778 sec","job_status":"done","queue_duration_s":1.411009,"scheduling_latency_s":1.411023,"redis_calls":13,"redis_duration_s":0.0017519999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009778,"completed_at":"2026-11-10T09:53:03.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608gitlab # [ 195.097746] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52e3a1d5dc0ee37128fdf3a2","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52e3a1d5dc0ee37128fdf3a2: start","job_status":"start","queue_duration_s":1.421644,"scheduling_latency_s":1.421664}1609gitlab # [ 195.098679] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52e3a1d5dc0ee37128fdf3a2","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52e3a1d5dc0ee37128fdf3a2: done: 0.009484 sec","job_status":"done","queue_duration_s":1.421644,"scheduling_latency_s":1.421664,"redis_calls":13,"redis_duration_s":0.0016029999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009484,"completed_at":"2026-11-10T09:53:03.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610gitlab # [ 195.137668] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8211612cbcd060ce7f0cb6b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8211612cbcd060ce7f0cb6b: start","job_status":"start","queue_duration_s":1.432,"scheduling_latency_s":1.432013}1611gitlab # [ 195.138568] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8211612cbcd060ce7f0cb6b","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8211612cbcd060ce7f0cb6b: done: 0.009277 sec","job_status":"done","queue_duration_s":1.432,"scheduling_latency_s":1.432013,"redis_calls":13,"redis_duration_s":0.001538,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009277,"completed_at":"2026-11-10T09:53:03.225Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612gitlab # [ 195.182635] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"653b06fdbcffded24fc3db7a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-653b06fdbcffded24fc3db7a: start","job_status":"start","queue_duration_s":1.442576,"scheduling_latency_s":1.442591}1613gitlab # [ 195.183591] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"653b06fdbcffded24fc3db7a","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-653b06fdbcffded24fc3db7a: done: 0.009669 sec","job_status":"done","queue_duration_s":1.442576,"scheduling_latency_s":1.442591,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009669,"completed_at":"2026-11-10T09:53:03.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614gitlab # [ 195.229079] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7b5859dccadf0ecacfdca45","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7b5859dccadf0ecacfdca45: start","job_status":"start","queue_duration_s":1.453091,"scheduling_latency_s":1.453104}1615gitlab # [ 195.229789] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7b5859dccadf0ecacfdca45","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7b5859dccadf0ecacfdca45: done: 0.009837 sec","job_status":"done","queue_duration_s":1.453091,"scheduling_latency_s":1.453104,"redis_calls":13,"redis_duration_s":0.00169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009837,"completed_at":"2026-11-10T09:53:03.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616gitlab # [ 195.279299] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c232ad14f31d6128e9a2f941","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c232ad14f31d6128e9a2f941: start","job_status":"start","queue_duration_s":1.463809,"scheduling_latency_s":1.463822}1617gitlab # [ 195.280095] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c232ad14f31d6128e9a2f941","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c232ad14f31d6128e9a2f941: done: 0.009371 sec","job_status":"done","queue_duration_s":1.463809,"scheduling_latency_s":1.463822,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009371,"completed_at":"2026-11-10T09:53:03.257Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618gitlab # [ 195.323895] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26460237098571904c532af9","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26460237098571904c532af9: start","job_status":"start","queue_duration_s":1.474483,"scheduling_latency_s":1.474497}1619gitlab # [ 195.324760] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26460237098571904c532af9","created_at":"2026-11-10T09:53:01.736Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-faef07421b3641fb","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"81d3353109f3b33196adb3ec6cb50a64","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.783Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26460237098571904c532af9: done: 0.009359 sec","job_status":"done","queue_duration_s":1.474483,"scheduling_latency_s":1.474497,"redis_calls":13,"redis_duration_s":0.001923,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009359,"completed_at":"2026-11-10T09:53:03.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620gitlab # [ 195.367651] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"ec5a7e15f8ae859fd2d1f02b","created_at":"2026-11-10T09:53:01.855Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-7dbe0e2bc4e24c83","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"fe20bbbca0c4a7be8a513fd29aa21099","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.861Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-ec5a7e15f8ae859fd2d1f02b: start","job_status":"start","queue_duration_s":1.407255,"scheduling_latency_s":1.407269}1621gitlab # [ 195.368332] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"ec5a7e15f8ae859fd2d1f02b","created_at":"2026-11-10T09:53:01.855Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-7dbe0e2bc4e24c83","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"fe20bbbca0c4a7be8a513fd29aa21099","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:01.861Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-ec5a7e15f8ae859fd2d1f02b: done: 0.027787 sec","job_status":"done","queue_duration_s":1.407255,"scheduling_latency_s":1.407269,"redis_calls":13,"redis_duration_s":0.001844,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210921,"duration_s":0.027787,"completed_at":"2026-11-10T09:53:03.296Z","load_balancing_strategy":"primary","db_duration_s":0.008917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622gitlab # [ 195.413584] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.297Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ed8727db830743c0e54b9e9c","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ed8727db830743c0e54b9e9c: start","job_status":"start","queue_duration_s":0.63613,"scheduling_latency_s":0.636144}1623gitlab # [ 195.414401] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.305Z","message":"No existing merge request to be cleaned up."}1624gitlab # [ 195.415064] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.310Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ed8727db830743c0e54b9e9c","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ed8727db830743c0e54b9e9c: done: 0.012983 sec","job_status":"done","queue_duration_s":0.63613,"scheduling_latency_s":0.636144,"redis_calls":10,"redis_duration_s":0.0013289999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000767,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012983,"completed_at":"2026-11-10T09:53:03.310Z","load_balancing_strategy":"primary","db_duration_s":0.002074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1625gitlab # [ 195.464611] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.311Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b5b5d46692d3c6e9b0c61403","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b5b5d46692d3c6e9b0c61403: start","job_status":"start","queue_duration_s":0.649982,"scheduling_latency_s":0.649997}1626gitlab # [ 195.465215] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.320Z","message":"No existing merge request to be cleaned up."}1627gitlab # [ 195.465745] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.324Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b5b5d46692d3c6e9b0c61403","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b5b5d46692d3c6e9b0c61403: done: 0.013261 sec","job_status":"done","queue_duration_s":0.649982,"scheduling_latency_s":0.649997,"redis_calls":10,"redis_duration_s":0.0033949999999999996,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00224,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013261,"completed_at":"2026-11-10T09:53:03.324Z","load_balancing_strategy":"primary","db_duration_s":0.001076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628gitlab # [ 195.507298] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.325Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3af4650fa83072f7b0c994e5","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3af4650fa83072f7b0c994e5: start","job_status":"start","queue_duration_s":0.664119,"scheduling_latency_s":0.664135}1629gitlab # [ 195.507965] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.332Z","message":"No existing merge request to be cleaned up."}1630gitlab # [ 195.508601] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.337Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3af4650fa83072f7b0c994e5","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3af4650fa83072f7b0c994e5: done: 0.01185 sec","job_status":"done","queue_duration_s":0.664119,"scheduling_latency_s":0.664135,"redis_calls":10,"redis_duration_s":0.001353,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01185,"completed_at":"2026-11-10T09:53:03.337Z","load_balancing_strategy":"primary","db_duration_s":0.001003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1631gitlab # [ 195.546278] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.338Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"02d19920d5a2663b644e85c0","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-02d19920d5a2663b644e85c0: start","job_status":"start","queue_duration_s":0.6769,"scheduling_latency_s":0.676914}1632gitlab # [ 195.546931] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.348Z","message":"No existing merge request to be cleaned up."}1633gitlab # [ 195.547586] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:03.353Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"02d19920d5a2663b644e85c0","created_at":"2026-11-10T09:53:02.656Z","trace_propagation_headers":{"sentry-trace":"d0cd31b0d6ec44a89ce5a935f180e0cb-61d53a07c786411f","baggage":"sentry-trace_id=d0cd31b0d6ec44a89ce5a935f180e0cb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e0637535215c96dfb82676ac50efd100","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:02.661Z","job_size_bytes":2,"pid":1653,"sidekiq_tid":"mae9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-02d19920d5a2663b644e85c0: done: 0.015608 sec","job_status":"done","queue_duration_s":0.6769,"scheduling_latency_s":0.676914,"redis_calls":10,"redis_duration_s":0.001446,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015608,"completed_at":"2026-11-10T09:53:03.353Z","load_balancing_strategy":"primary","db_duration_s":0.00694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634gitlab # [ 195.932623] runuser[2279]: pam_unix(runuser:session): session closed for user gitlab1635gitlab # [ 195.935997] setup-gitlab-start[2257]: + systemctl restart gitlab-sidekiq.service1636gitlab # [ 195.959593] systemd[1]: Stopping gitlab-sidekiq.service...1637gitlab # [ 195.964092] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:06.188Z","message":"Shutting down"}1638gitlab # [ 195.965627] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:06.189Z","message":"Scheduler exiting..."}1639gitlab # [ 195.967099] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:06.189Z","message":"Terminating quiet threads for default capsule"}1640gitlab # [ 195.968265] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:06.189Z","message":"Scheduler exiting..."}1641gitlab # [ 196.467268] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:06.690Z","message":"Pausing to allow jobs to finish..."}1642gitlab # [ 198.470911] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:08.694Z","message":"Bye!"}1643gitlab # [ 198.499999] sidekiq-cluster[1653]: {"severity":"INFO","time":"2026-11-10T09:53:08.724Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1653,"worker_id":"sidekiq_0","memwd_rss_bytes":945311744,"retry":0}1644gitlab # [ 199.264213] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1645gitlab # [ 199.265424] systemd[1]: Stopped gitlab-sidekiq.service.1646gitlab # [ 199.266666] systemd[1]: gitlab-sidekiq.service: Consumed 1min 3.827s CPU time over 2min 2.150s wall clock time, 957.8M memory peak, 4.5M read from disk.1647gitlab # [ 199.270099] systemd[1]: Started gitlab-sidekiq.service.1648gitlab # [ 199.274679] setup-gitlab-start[2257]: + mkdir -p /var/lib/secrets1649gitlab # [ 199.279849] setup-gitlab-start[2257]: + echo nixbot-test-token-of-20-chars1650gitlab # [ 199.280937] setup-gitlab-start[2257]: + chmod 644 /var/lib/secrets/gitlab-token1651gitlab # [ 199.283555] setup-gitlab-start[2257]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1652gitlab # [ 199.867904] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:10 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 5491653gitlab # [ 200.447735] gitaly[911]: time="2026-11-10T09:53:10.671Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.671" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654gitlab # [ 200.449886] gitaly[911]: time="2026-11-10T09:53:10.672Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.671" grpc.time_ms=0.33 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1655gitlab # [ 200.543351] gitaly[911]: time="2026-11-10T09:53:10.768Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.097" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.767" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1656gitlab # [ 200.544875] gitaly[911]: time="2026-11-10T09:53:10.768Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.097" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.767" grpc.time_ms=0.404 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1657gitlab # [ 200.606434] gitaly[911]: time="2026-11-10T09:53:10.831Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 diskcache=4c7f5715-cba4-4ac7-8ee3-536c965dd615 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T09:53:20.105" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.775" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1658gitlab # [ 200.607848] gitaly[911]: time="2026-11-10T09:53:10.831Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1573 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T09:53:20.105" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.775" grpc.time_ms=55.763 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1659gitlab # [ 200.619662] gitaly[911]: time="2026-11-10T09:53:10.844Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.173" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.843" pid=911 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1660gitlab # [ 200.620906] gitaly[911]: time="2026-11-10T09:53:10.844Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.173" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.843" grpc.time_ms=0.89 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1661gitlab # [ 200.634173] gitaly[911]: time="2026-11-10T09:53:10.859Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:53:20.177" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.847" grpc.time_ms=10.894 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1662gitlab # [ 200.709926] gitaly[911]: time="2026-11-10T09:53:10.934Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.263" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.934" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1663gitlab # [ 200.711198] gitaly[911]: time="2026-11-10T09:53:10.936Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:20.263" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.934" grpc.time_ms=0.447 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 200.765831] gitaly[911]: time="2026-11-10T09:53:10.990Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 diskcache=8b066f83-1a72-4a72-91c6-ea5d67044d8b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T09:53:20.268" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.939" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1665gitlab # [ 200.766940] gitaly[911]: time="2026-11-10T09:53:10.990Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1575 command.oublock=32 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T09:53:20.268" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:10.939" grpc.time_ms=51.795 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1666gitlab # [ 200.796750] gitaly[911]: time="2026-11-10T09:53:11.021Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:21.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:11.021" pid=911 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1667gitlab # [ 200.798827] gitaly[911]: time="2026-11-10T09:53:11.021Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:21.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:11.021" grpc.time_ms=0.431 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1668gitlab # [ 200.814229] gitaly[911]: time="2026-11-10T09:53:11.039Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:53:21.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:11.028" grpc.time_ms=10.376 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1669gitlab # [ 200.863289] gitaly[911]: time="2026-11-10T09:53:11.088Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:53:21.067" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:11.087" grpc.time_ms=0.257 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670gitlab # [ 200.873446] gitaly[911]: time="2026-11-10T09:53:11.095Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T09:53:21.074" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:11.095" grpc.time_ms=0.424 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671gitlab # [ 201.312327] gitaly[911]: time="2026-11-10T09:53:11.537Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0CZ13VXXBMH54J57WNE0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:53:21.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:11.526" grpc.time_ms=10.967 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1672gitlab # [ 201.698362] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:11 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 18081673gitlab # [ 201.701381] setup-gitlab-start[2531]: {"id":1,"message":null,"error":null}1674gitlab # [ 202.596465] sidekiq-cluster[2518]: {"severity":"INFO","time":"2026-11-10T09:53:12.802Z","message":"Starting cluster with 1 processes"}1675gitlab # [ 206.829456] gitaly[911]: time="2026-11-10T09:53:17.053Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0KMK8H7R7CV63ZQXRY36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:53:27.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:17.041" grpc.time_ms=11.532 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1676gitlab # [ 206.871701] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:17 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1471677gitlab # [ 206.878410] setup-gitlab-start[2257]: + mkdir -p /tmp/test-flake1678gitlab # [ 206.883442] setup-gitlab-start[2257]: + cd /tmp/test-flake1679gitlab # [ 206.885283] setup-gitlab-start[2257]: + git init -b master1680gitlab # [ 206.913813] setup-gitlab-start[2603]: Initialized empty Git repository in /tmp/test-flake/.git/1681gitlab # [ 206.916723] setup-gitlab-start[2257]: + git config user.name test1682gitlab # [ 206.918868] setup-gitlab-start[2257]: + git config user.email test@example.com1683gitlab # [ 206.922222] setup-gitlab-start[2257]: + cat1684gitlab # [ 206.925772] setup-gitlab-start[2257]: + git add flake.nix1685gitlab # [ 206.930918] setup-gitlab-start[2257]: + git commit -m 'initial commit'1686gitlab # [ 206.953241] setup-gitlab-start[2608]: [master (root-commit) 64ce515] initial commit1687gitlab # [ 206.954023] setup-gitlab-start[2608]: 1 file changed, 10 insertions(+)1688gitlab # [ 206.954708] setup-gitlab-start[2608]: create mode 100644 flake.nix1689gitlab # [ 206.956093] setup-gitlab-start[2257]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1690gitlab # [ 206.960165] setup-gitlab-start[2257]: + git push -u origin master1691gitlab # [ 207.078314] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 951692gitlab # [ 207.309059] gitaly[911]: time="2026-11-10T09:53:17.533Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0KZXMQ6HWP3RRY7FDMV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:17.520" grpc.time_ms=12.738 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1693gitlab # [ 207.341214] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 2551694gitlab # [ 207.640283] gitaly[911]: time="2026-11-10T09:53:17.864Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:53:27.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:17.853" grpc.time_ms=11.362 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1695gitlab # [ 207.677375] gitaly[911]: time="2026-11-10T09:53:17.902Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T09:53:47.089" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T09:53:17.889" grpc.time_ms=12.228 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1696gitlab # [ 207.706144] gitaly[911]: time="2026-11-10T09:53:17.921Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV duration_ms=150ns method=POST pid=911 status=200 url="http://unix/api/v4/internal/allowed"1697gitlab # [ 207.731094] gitaly[911]: time="2026-11-10T09:53:17.955Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV duration_ms=33ns method=POST pid=911 status=200 url="http://unix/api/v4/internal/pre_receive"1698gitlab # [ 207.737641] gitaly[911]: time="2026-11-10T09:53:17.956Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5311 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:17.769" grpc.time_ms=186.66 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1699gitlab # [ 207.777529] gitaly[911]: time="2026-11-10T09:53:18.002Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:18.001" grpc.time_ms=0.551 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1700gitlab # [ 207.817817] gitaly[911]: time="2026-11-10T09:53:18.042Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:18.042" grpc.time_ms=0.42 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1701gitlab # [ 207.849924] gitaly[911]: time="2026-11-10T09:53:18.074Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:18.074" grpc.time_ms=0.266 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1702gitlab # [ 207.965046] gitaly[911]: time="2026-11-10T09:53:18.189Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:53:28.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:18.178" grpc.time_ms=11.361 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1703gitlab # [ 208.021955] gitaly[911]: time="2026-11-10T09:53:18.246Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1563 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T09:53:28.053" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:18.224" grpc.time_ms=22.471 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704gitlab # [ 208.054431] gitaly[911]: time="2026-11-10T09:53:18.279Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV duration_ms=175ns method=POST pid=911 status=200 url="http://unix/api/v4/internal/post_receive"1705gitlab # [ 208.057672] gitaly[911]: time="2026-11-10T09:53:18.282Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4805 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:18.103" grpc.time_ms=178.932 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1706gitlab # [ 208.077492] gitaly[911]: time="2026-11-10T09:53:18.302Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV diskcache=ab5dc148-4771-49d5-8206-da4be8956639 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:17.682" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1707gitlab # [ 208.078050] gitaly[911]: time="2026-11-10T09:53:18.302Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=208 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13794 command.oublock=32 command.real_time_ms=610 command.spawn_token_fork_ms=0 command.system_time_ms=139 command.user_time_ms=69 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:17.682" grpc.time_ms=620.203 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1708gitlab # [ 208.079131] gitlab-workhorse[825]: time="2026-11-10T09:53:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521709gitlab # [ 208.079672] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:18 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7231710gitlab # [ 208.082172] setup-gitlab-start[2612]: To http://gitlab/root/test-flake.git1711gitlab # [ 208.083086] setup-gitlab-start[2612]: * [new branch] master -> master1712gitlab # [ 208.106098] setup-gitlab-start[2612]: branch 'master' set up to track 'origin/master'.1713gitlab # [ 208.109978] systemd[1]: Finished setup-gitlab.service.1714gitlab # [ 208.117608] systemd[1]: Starting nixbot CI service...1715gitlab: (finished: waiting for unit setup-gitlab.service, in 209.18 seconds)1716gitlab: waiting for unit nixbot.service1717gitlab # [ 209.524636] nixbot[2700]: {"ts": "2026-11-10T09:53:19.748Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1718gitlab # [ 209.569562] nixbot[2700]: {"ts": "2026-11-10T09:53:19.794Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1719gitlab # [ 209.595552] nixbot[2700]: {"ts": "2026-11-10T09:53:19.820Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1720gitlab # [ 209.601766] nixbot[2700]: {"ts": "2026-11-10T09:53:19.826Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1721gitlab # [ 209.607726] nixbot[2700]: {"ts": "2026-11-10T09:53:19.832Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1722gitlab # [ 209.615234] nixbot[2700]: {"ts": "2026-11-10T09:53:19.840Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1723gitlab # [ 209.621174] nixbot[2700]: {"ts": "2026-11-10T09:53:19.845Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1724gitlab # [ 209.625907] nixbot[2700]: {"ts": "2026-11-10T09:53:19.850Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1725gitlab # [ 209.637720] nixbot[2700]: {"ts": "2026-11-10T09:53:19.862Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1726gitlab # [ 209.643791] nixbot[2700]: {"ts": "2026-11-10T09:53:19.868Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1727gitlab # [ 209.650852] nixbot[2700]: {"ts": "2026-11-10T09:53:19.875Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1728gitlab # [ 209.662126] nixbot[2700]: {"ts": "2026-11-10T09:53:19.886Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1729gitlab # [ 209.667662] nixbot[2700]: {"ts": "2026-11-10T09:53:19.892Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1730gitlab # [ 209.673093] nixbot[2700]: {"ts": "2026-11-10T09:53:19.897Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1731gitlab # [ 209.680922] nixbot[2700]: {"ts": "2026-11-10T09:53:19.905Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1732gitlab # [ 209.686238] nixbot[2700]: {"ts": "2026-11-10T09:53:19.910Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1733gitlab # [ 209.690534] nixbot[2700]: {"ts": "2026-11-10T09:53:19.915Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1734gitlab # [ 209.694783] nixbot[2700]: {"ts": "2026-11-10T09:53:19.919Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1735gitlab # [ 209.701242] nixbot[2700]: {"ts": "2026-11-10T09:53:19.926Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1736gitlab # [ 209.705499] nixbot[2700]: {"ts": "2026-11-10T09:53:19.930Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1737gitlab # [ 209.709662] nixbot[2700]: {"ts": "2026-11-10T09:53:19.934Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1738gitlab # [ 209.718162] nixbot[2700]: {"ts": "2026-11-10T09:53:19.942Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1739gitlab # [ 209.723103] nixbot[2700]: {"ts": "2026-11-10T09:53:19.947Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1740gitlab # [ 209.727969] nixbot[2700]: {"ts": "2026-11-10T09:53:19.952Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1741gitlab # [ 209.732179] nixbot[2700]: {"ts": "2026-11-10T09:53:19.956Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1742gitlab # [ 209.737085] nixbot[2700]: {"ts": "2026-11-10T09:53:19.961Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1743gitlab # [ 209.741622] nixbot[2700]: {"ts": "2026-11-10T09:53:19.966Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1744gitlab # [ 209.745653] nixbot[2700]: {"ts": "2026-11-10T09:53:19.970Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1745gitlab # [ 209.749979] nixbot[2700]: {"ts": "2026-11-10T09:53:19.974Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1746gitlab # [ 209.768273] nixbot[2700]: {"ts": "2026-11-10T09:53:19.993Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1747gitlab # [ 209.776870] nixbot[2700]: {"ts": "2026-11-10T09:53:20.001Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1748gitlab # [ 209.781774] nixbot[2700]: {"ts": "2026-11-10T09:53:20.006Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1749gitlab # [ 209.786525] nixbot[2700]: {"ts": "2026-11-10T09:53:20.011Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1750gitlab # [ 209.793119] nixbot[2700]: {"ts": "2026-11-10T09:53:20.017Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1751gitlab # [ 209.797319] nixbot[2700]: {"ts": "2026-11-10T09:53:20.022Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1752gitlab # [ 209.804100] nixbot[2700]: {"ts": "2026-11-10T09:53:20.028Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1753gitlab # [ 209.809442] nixbot[2700]: {"ts": "2026-11-10T09:53:20.034Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1754gitlab # [ 209.818331] nixbot[2700]: {"ts": "2026-11-10T09:53:20.043Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1755gitlab # [ 209.823677] nixbot[2700]: {"ts": "2026-11-10T09:53:20.048Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1756gitlab # [ 210.225941] nixbot[2700]: {"ts": "2026-11-10T09:53:20.450Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2700]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1757gitlab # [ 210.228824] nixbot[2700]: {"ts": "2026-11-10T09:53:20.451Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1758gitlab # [ 210.250194] nixbot[2700]: {"ts": "2026-11-10T09:53:20.475Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:56836 - \"GET /health HTTP/1.1\" 200"}1759gitlab # [ 210.263512] systemd[1]: Started nixbot CI service.1760gitlab # [ 210.267249] systemd[1]: Reached target Multi-User System.1761gitlab # [ 210.268247] systemd[1]: Startup finished in 1.128s (kernel) + 3.634s (initrd) + 3min 25.505s (userspace) = 3min 30.268s.1762gitlab: (finished: waiting for unit nixbot.service, in 2.08 seconds)1763gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1764gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds)1765subtest: project discovered and webhook registered1766gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1767gitlab # [ 210.603772] nixbot[2700]: {"ts": "2026-11-10T09:53:20.827Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:56842 - \"GET /health HTTP/1.1\" 200"}1768gitlab # [ 210.669990] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 461769gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds)1770gitlab # [ 210.934145] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 6801771gitlab # [ 210.956232] nixbot[2700]: {"ts": "2026-11-10T09:53:21.180Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1772gitlab # [ 211.010790] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 391773gitlab # [ 211.016418] nixbot[2700]: {"ts": "2026-11-10T09:53:21.241Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1774gitlab # [ 211.212586] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1921775gitlab # [ 211.364962] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1401776gitlab # [ 211.490273] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 1151777gitlab # [ 211.568161] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 711778gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1779gitlab # [ 211.655886] gitaly[911]: time="2026-11-10T09:53:21.880Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RC6P2WV4WX41FV9HBYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-10T09:53:51.060" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:21.861" grpc.time_ms=17.818 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 211.689599] gitaly[911]: time="2026-11-10T09:53:21.895Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RC6P2WV4WX41FV9HBYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:53:51.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:21.894" grpc.time_ms=0.919 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab # [ 211.712250] gitaly[911]: time="2026-11-10T09:53:21.937Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=12 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RC6P2WV4WX41FV9HBYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-10T09:53:31.069" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:21.919" grpc.time_ms=17.313 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782gitlab # [ 211.733741] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1591783gitlab # [ 211.768609] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 451784gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds)1785(finished: subtest: project discovered and webhook registered, in 1.17 seconds)1786subtest: push delivers the webhook1787gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1788gitlab # [ 211.802481] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 641789gitlab # [ 211.807542] nixbot[2700]: {"ts": "2026-11-10T09:53:22.030Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1790gitlab # [ 211.912162] gitaly[911]: time="2026-11-10T09:53:22.136Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0RN47K14AP25VAE554VF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.125" grpc.time_ms=9.993 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1791gitlab # [ 211.940235] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 521792gitlab # [ 212.000559] gitaly[911]: time="2026-11-10T09:53:22.225Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RPVNJVGANX6PE6BJ2QM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.211" pid=911 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1793gitlab # [ 212.015907] gitaly[911]: time="2026-11-10T09:53:22.240Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RPVNJVGANX6PE6BJ2QM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.211" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1794gitlab # [ 212.027973] gitaly[911]: time="2026-11-10T09:53:22.252Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RPVNJVGANX6PE6BJ2QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.211" grpc.time_ms=40.813 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1795gitlab # [ 212.041838] gitlab-workhorse[825]: time="2026-11-10T09:53:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131796gitlab # [ 212.043633] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1131797gitlab # [ 212.084895] gitaly[911]: time="2026-11-10T09:53:22.309Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RTH6F6VZKQWBFB2EY8F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.297" pid=911 remote_ip="2001:db8:1::1" request_sha=2e2f26e8d3e00057c593046f84921a1e1d3b98a1 response_bytes=223 span.kind=server system=grpc1798gitlab # [ 212.086607] gitaly[911]: time="2026-11-10T09:53:22.309Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RTH6F6VZKQWBFB2EY8F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.297" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1799gitlab # [ 212.099297] gitlab-workhorse[825]: time="2026-11-10T09:53:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231800gitlab # [ 212.099929] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 531801gitlab # [ 212.100600] gitaly[911]: time="2026-11-10T09:53:22.310Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0RTH6F6VZKQWBFB2EY8F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.297" grpc.time_ms=12.783 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1802gitlab # [ 212.196547] gitaly[911]: time="2026-11-10T09:53:22.421Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0RY99KRJ0GN3WH4NJ14H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.411" grpc.time_ms=9.936 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1803gitlab # [ 212.197348] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 441804gitlab # [ 212.271186] gitaly[911]: time="2026-11-10T09:53:22.496Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0S0DQ96F3H18RWDFEECY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.487" pid=911 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1805gitlab # [ 212.290862] gitaly[911]: time="2026-11-10T09:53:22.496Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0S0DQ96F3H18RWDFEECY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.487" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1806gitlab # [ 212.307922] gitaly[911]: time="2026-11-10T09:53:22.532Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0S0DQ96F3H18RWDFEECY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.487" grpc.time_ms=28.55 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1807gitlab # [ 212.321335] gitlab-workhorse[825]: time="2026-11-10T09:53:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671808gitlab # [ 212.322277] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1001809gitlab # [ 212.409499] gitaly[911]: time="2026-11-10T09:53:22.634Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0S4V8N7QN845BG2KE7RC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.626" grpc.time_ms=7.566 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1810gitlab # [ 212.432548] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691811gitlab # [ 212.498389] gitaly[911]: time="2026-11-10T09:53:22.723Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0S73K0YHDXEB74968WSF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.705" pid=911 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1812gitlab # [ 212.520228] gitaly[911]: time="2026-11-10T09:53:22.723Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0S73K0YHDXEB74968WSF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.705" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1813gitlab # [ 212.522428] gitaly[911]: time="2026-11-10T09:53:22.745Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0S73K0YHDXEB74968WSF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:22.705" grpc.time_ms=39.882 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1814gitlab # [ 212.557777] gitlab-workhorse[825]: time="2026-11-10T09:53:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091815gitlab # [ 212.559082] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1221816gitlab # [ 212.668400] gitaly[911]: time="2026-11-10T09:53:22.892Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:53:52.091" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:22.891" grpc.time_ms=0.773 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 212.705410] gitaly[911]: time="2026-11-10T09:53:22.930Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T09:53:52.120" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:22.920" grpc.time_ms=9.439 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 212.743053] gitaly[911]: time="2026-11-10T09:53:22.967Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:53:32.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:22.958" grpc.time_ms=9.756 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab # [ 212.758937] gitaly[911]: time="2026-11-10T09:53:22.983Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:53:32.024" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:22.974" grpc.time_ms=9.055 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab # [ 212.917169] gitaly[911]: time="2026-11-10T09:53:23.141Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T09:53:33.121" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:23.141" grpc.time_ms=0.227 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 212.941073] gitaly[911]: time="2026-11-10T09:53:23.163Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T09:53:33.133" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:23.153" grpc.time_ms=10.002 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 212.995945] gitaly[911]: time="2026-11-10T09:53:23.220Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:23.220" grpc.time_ms=0.304 pid=911 span.kind=server system=grpc1823gitlab # [ 213.041170] gitaly[911]: time="2026-11-10T09:53:23.266Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:53:23.265" grpc.time_ms=0.327 pid=911 span.kind=server system=grpc1824gitlab # [ 213.062040] gitaly[911]: time="2026-11-10T09:53:23.286Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE diskcache=94233077-b580-40ae-ab9f-c7d9e2f901af grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:53:33.146" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:23.166" pid=911 ref=refs/keep-around/64ce515318877fcd7b1ffb60d5f22da640a32041 remote_ip="2001:db8:1::1" revision=64ce515318877fcd7b1ffb60d5f22da640a32041 span.kind=server system=grpc username=root1825gitlab # [ 213.072583] gitaly[911]: time="2026-11-10T09:53:23.287Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=70 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5343 command.oublock=8 command.real_time_ms=109 command.spawn_token_fork_ms=25 command.system_time_ms=43 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:53:33.146" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:53:23.166" grpc.time_ms=120.548 pid=911 ref=refs/keep-around/64ce515318877fcd7b1ffb60d5f22da640a32041 remote_ip="2001:db8:1::1" revision=64ce515318877fcd7b1ffb60d5f22da640a32041 span.kind=server system=grpc username=root1826gitlab # [ 213.085809] gitaly[911]: time="2026-11-10T09:53:23.292Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:53:53.091" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:23.291" grpc.time_ms=0.445 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827gitlab # [ 213.294789] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6921828gitlab # [ 213.311327] nixbot[2700]: {"ts": "2026-11-10T09:53:23.535Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-64ce51531887-0"}1829gitlab # [ 213.517365] nixbot[2700]: {"ts": "2026-11-10T09:53:23.742Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-64ce51531887-0", "argv0": "bwrap"}1830gitlab # [ 213.622224] systemd[1]: Started Nix Daemon.1831gitlab # [ 213.698099] nix-daemon[2865]: accepted connection from pid 2860, user nixbot1832gitlab # [ 218.639649] nix-daemon[2865]: accepted connection from pid 2914, user nixbot1833gitlab # [ 218.657888] nix-daemon[2865]: accepted connection from pid 2860, user nixbot1834gitlab # [ 222.775074] nix-daemon[2865]: accepted connection from pid 2860, user nixbot1835gitlab # [ 222.867059] nix-daemon[2865]: accepted connection from pid 2956, user nixbot1836gitlab # [ 222.931511] gitaly[911]: time="2026-11-10T09:53:33.154Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E13CRJHB73H7MS10HX3RQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:03.052" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:33.153" grpc.time_ms=1.009 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 223.088085] gitaly[911]: time="2026-11-10T09:53:33.312Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E13CRJHB73H7MS10HX3RQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:53:43.131" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:33.301" grpc.time_ms=11.21 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 223.167306] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 3091839gitlab # [ 223.211129] gitaly[911]: time="2026-11-10T09:53:33.436Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E13PRNN0MAF303DFYFN6K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:03.334" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:33.435" grpc.time_ms=0.858 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 223.278036] gitaly[911]: time="2026-11-10T09:53:33.502Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E13PRNN0MAF303DFYFN6K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:53:43.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:33.492" grpc.time_ms=9.872 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841gitlab # [ 223.334509] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1581842gitlab # [ 227.585437] nix-daemon[2865]: accepted connection from pid 3028, user nixbot1843gitlab # [ 227.687181] gitaly[911]: time="2026-11-10T09:53:37.911Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E181NQFF02T6ZDKDK3SJ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:07.109" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:53:37.910" grpc.time_ms=0.787 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844gitlab # [ 227.750829] gitaly[911]: time="2026-11-10T09:53:37.975Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E181NQFF02T6ZDKDK3SJ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:53:47.114" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:53:37.964" grpc.time_ms=10.71 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 227.838159] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2151846gitlab # [ 228.113988] nixbot[2700]: {"ts": "2026-11-10T09:53:38.337Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "64ce515318877fcd7b1ffb60d5f22da640a32041", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1847gitlab # [ 228.230183] gitaly[911]: time="2026-11-10T09:53:38.455Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72E18JM5D36ZE3FWMWKEEK4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.443" grpc.time_ms=11.479 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1848gitlab # [ 228.242411] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 661849gitlab # [ 228.298847] gitaly[911]: time="2026-11-10T09:53:38.523Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E18MTVP9KKPGZW2K4ZTS2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.514" pid=911 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1850gitlab # [ 228.300561] gitaly[911]: time="2026-11-10T09:53:38.523Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E18MTVP9KKPGZW2K4ZTS2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.514" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1851gitlab # [ 228.339399] gitaly[911]: time="2026-11-10T09:53:38.563Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E18MTVP9KKPGZW2K4ZTS2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.514" grpc.time_ms=49.513 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1852gitlab # [ 228.340667] gitlab-workhorse[825]: time="2026-11-10T09:53:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671853gitlab # [ 228.341528] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1051854gitlab # [ 228.417690] gitaly[911]: time="2026-11-10T09:53:38.642Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E18S7R96XGXEMKCWB8RW9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.637" grpc.time_ms=5.239 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1855gitlab # [ 228.429920] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 541856gitlab # [ 228.472965] gitaly[911]: time="2026-11-10T09:53:38.697Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E18V0Y4DT8JPB7EQMQFJD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.692" pid=911 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1857gitlab # [ 228.474128] gitaly[911]: time="2026-11-10T09:53:38.698Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E18V0Y4DT8JPB7EQMQFJD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.692" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1858gitlab # [ 228.474810] gitlab-workhorse[825]: time="2026-11-10T09:53:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671859gitlab # [ 228.475394] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:53:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 421860gitlab # [ 228.475890] gitaly[911]: time="2026-11-10T09:53:38.698Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E18V0Y4DT8JPB7EQMQFJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:53:38.692" grpc.time_ms=6.013 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1861gitlab # [ 229.166267] sidekiq-cluster[2576]: 2026-11-10T09:53:39.390Z pid=2576 tid=xbs INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2576, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>439595008}1862gitlab # [ 229.356615] sidekiq-cluster[2576]: W, [2026-11-10T09:53:39.581447 #2576] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1863gitlab # [ 229.397794] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:53:39.622Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1864gitlab # [ 231.225745] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:53:41.450Z","message":"GitLab reliable fetch activated!"}1865gitlab # [ 231.226695] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:53:41.450Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1866gitlab # [ 253.581202] postgres[3281]: [3281] ERROR: relation "table_without_models" does not exist1867gitlab # [ 253.582509] postgres[3281]: [3281] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1868gitlab # [ 256.213530] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:06.437Z","message":"Booted Rails 7.2.3.1 application in production environment"}1869gitlab # [ 256.215631] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:06.439Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1870gitlab # [ 256.216649] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:06.439Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1871gitlab # [ 256.217774] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:06.439Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1872gitlab # [ 256.218917] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:06.441Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1873gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 45.43 seconds)1874gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1875gitlab # [ 257.326082] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:07 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 591876gitlab # [ 257.414957] gitaly[911]: time="2026-11-10T09:54:07.639Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72E2524CY288ZX7PV4MTG5Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:07.626" grpc.time_ms=12.459 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1877gitlab # [ 257.416504] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:07 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 821878gitlab # Enumerating objects: 5, done.1879gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1880gitlab # Delta compression using up to 4 threads1881gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1882gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1883gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1884gitlab # [ 257.691321] gitaly[911]: time="2026-11-10T09:54:07.916Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T09:54:37.708" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T09:54:07.908" grpc.time_ms=7.305 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 257.707862] gitaly[911]: time="2026-11-10T09:54:07.932Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72E255ZJ2VBBPT028CKWW3Z duration_ms=88ns method=POST pid=911 status=200 url="http://unix/api/v4/internal/allowed"1886gitlab # [ 257.731533] gitaly[911]: time="2026-11-10T09:54:07.956Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72E255ZJ2VBBPT028CKWW3Z duration_ms=21ns method=POST pid=911 status=200 url="http://unix/api/v4/internal/pre_receive"1887gitlab # [ 257.738060] gitaly[911]: time="2026-11-10T09:54:07.963Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5311 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:07.843" grpc.time_ms=118.855 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1888gitlab # [ 257.791333] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.015Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0888f39d23d0605028cd8b7c","created_at":"2026-11-10T09:53:10.869Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:10.902Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0888f39d23d0605028cd8b7c: start","job_status":"start","queue_duration_s":57.112968,"scheduling_latency_s":57.113028}1889gitlab # [ 257.798212] gitaly[911]: time="2026-11-10T09:54:08.023Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:08.022" grpc.time_ms=0.524 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1890gitlab # [ 257.853817] gitaly[911]: time="2026-11-10T09:54:08.078Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:08.077" grpc.time_ms=0.678 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1891gitlab # [ 257.889603] gitaly[911]: time="2026-11-10T09:54:08.114Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:08.113" grpc.time_ms=0.426 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1892gitlab # [ 257.945037] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.169Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0888f39d23d0605028cd8b7c","created_at":"2026-11-10T09:53:10.869Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:10.902Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0888f39d23d0605028cd8b7c: done: 0.154018 sec","job_status":"done","queue_duration_s":57.112968,"scheduling_latency_s":57.113028,"redis_calls":24,"redis_duration_s":0.0044740000000000005,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.002092,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082661903,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.154018,"completed_at":"2026-11-10T09:54:08.169Z","load_balancing_strategy":"primary","db_duration_s":0.038917,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1893gitlab # [ 257.946656] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.171Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a56a2955761d2449182dabfb","created_at":"2026-11-10T09:53:10.920Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:10.927Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a56a2955761d2449182dabfb: start","job_status":"start","queue_duration_s":57.243131,"scheduling_latency_s":57.243157}1894gitlab # [ 258.024896] gitaly[911]: time="2026-11-10T09:54:08.249Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72E255ZJ2VBBPT028CKWW3Z duration_ms=105ns method=POST pid=911 status=200 url="http://unix/api/v4/internal/post_receive"1895gitlab # [ 258.028295] gitaly[911]: time="2026-11-10T09:54:08.253Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4805 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:08.143" grpc.time_ms=109.374 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1896gitlab # [ 258.035614] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.260Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a56a2955761d2449182dabfb","created_at":"2026-11-10T09:53:10.920Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:10.927Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a56a2955761d2449182dabfb: done: 0.08921 sec","job_status":"done","queue_duration_s":57.243131,"scheduling_latency_s":57.243157,"redis_calls":14,"redis_duration_s":0.002518,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0012,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000781,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.067584009,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.08921,"completed_at":"2026-11-10T09:54:08.260Z","load_balancing_strategy":"primary","db_duration_s":0.017208,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1897gitlab # [ 258.076103] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.262Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"3a74563dc56f852d5c2476e9","created_at":"2026-11-10T09:53:11.421Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:11.426Z","job_size_bytes":5,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-3a74563dc56f852d5c2476e9: start","job_status":"start","queue_duration_s":56.835618,"scheduling_latency_s":56.835643}1898gitlab # [ 258.105101] gitaly[911]: time="2026-11-10T09:54:08.330Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z diskcache=809d750f-0e56-4051-93d9-61f3d104a81b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:07.762" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1899gitlab # [ 258.106211] gitlab-workhorse[825]: time="2026-11-10T09:54:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521900gitlab # [ 258.106888] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:08 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6511901gitlab # [ 258.118883] gitaly[911]: time="2026-11-10T09:54:08.330Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=212 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13579 command.oublock=32 command.real_time_ms=558 command.spawn_token_fork_ms=0 command.system_time_ms=137 command.user_time_ms=75 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:07.762" grpc.time_ms=567.419 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1902gitlab # To http://gitlab/root/test-flake.git1903gitlab # 64ce515..a59288a master -> master1904gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.93 seconds)1905gitlab: must succeed: git -C /tmp/test-flake rev-parse master1906gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1907gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a'1908gitlab # [ 258.192655] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.417Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"3a74563dc56f852d5c2476e9","created_at":"2026-11-10T09:53:11.421Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:11.426Z","job_size_bytes":5,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-3a74563dc56f852d5c2476e9: done: 0.155043 sec","job_status":"done","queue_duration_s":56.835618,"scheduling_latency_s":56.835643,"redis_calls":12,"redis_duration_s":0.002166,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000517,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000296127,"duration_s":0.155043,"completed_at":"2026-11-10T09:54:08.417Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032552,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1909gitlab # [ 258.216814] nixbot[2700]: {"ts": "2026-11-10T09:54:08.441Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47488 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1\" 200"}1910gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', in 0.09 seconds)1911gitlab # [ 258.246331] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.470Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"4a7b3e7b18c9ac0f58aecc68","created_at":"2026-11-10T09:53:11.449Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:11.452Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-4a7b3e7b18c9ac0f58aecc68: start","job_status":"start","queue_duration_s":57.01796,"scheduling_latency_s":57.017995}1912gitlab # [ 258.424461] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.649Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"4a7b3e7b18c9ac0f58aecc68","created_at":"2026-11-10T09:53:11.449Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:11.452Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-4a7b3e7b18c9ac0f58aecc68: done: 0.178207 sec","job_status":"done","queue_duration_s":57.01796,"scheduling_latency_s":57.017995,"redis_calls":11,"redis_duration_s":0.001943,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.136,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.136,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000290261,"duration_s":0.178207,"completed_at":"2026-11-10T09:54:08.649Z","load_balancing_strategy":"primary","db_duration_s":0.062441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1913gitlab # [ 258.494062] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.718Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e562ab37ce594b95dbf09e3d","created_at":"2026-11-10T09:53:11.455Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:11.458Z","job_size_bytes":89,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e562ab37ce594b95dbf09e3d: start","job_status":"start","queue_duration_s":57.259913,"scheduling_latency_s":57.259945}1914gitlab # [ 258.562155] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.786Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"e562ab37ce594b95dbf09e3d","created_at":"2026-11-10T09:53:11.455Z","trace_propagation_headers":{"sentry-trace":"8718a767b6f94b98b0475bccb0b4d0c3-5d9101b15a194002","baggage":"sentry-trace_id=8718a767b6f94b98b0475bccb0b4d0c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0CZ13VXXBMH54J57WNE0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:11.458Z","job_size_bytes":89,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-e562ab37ce594b95dbf09e3d: done: 0.068007 sec","job_status":"done","queue_duration_s":57.259913,"scheduling_latency_s":57.259945,"redis_calls":11,"redis_duration_s":0.003657,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001063,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001991,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000539454,"duration_s":0.068007,"completed_at":"2026-11-10T09:54:08.786Z","load_balancing_strategy":"primary","db_duration_s":0.006789,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1915gitlab # [ 258.564135] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.788Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 64ce515318877fcd7b1ffb60d5f22da640a32041 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c15d539b39a6935a66044a4d","created_at":"2026-11-10T09:53:18.131Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-0ad81291b4f34ac1","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:18.152Z","job_size_bytes":185,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c15d539b39a6935a66044a4d: start","job_status":"start","queue_duration_s":50.635501,"scheduling_latency_s":50.635531}1916gitlab # [ 258.581926] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:08.800Z","message":"changes: \"0000000000000000000000000000000000000000 64ce515318877fcd7b1ffb60d5f22da640a32041 refs/heads/master\\n\""}1917gitlab # [ 258.606930] gitaly[911]: time="2026-11-10T09:54:08.831Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:18.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:08.820" grpc.time_ms=10.867 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 258.622115] gitaly[911]: time="2026-11-10T09:54:08.847Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:54:18.029" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:08.839" grpc.time_ms=7.386 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 258.674327] gitaly[911]: time="2026-11-10T09:54:08.899Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T09:54:18.080" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:08.891" grpc.time_ms=8.208 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 258.707117] gitaly[911]: time="2026-11-10T09:54:08.932Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3932 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:08.931" grpc.time_ms=0.293 pid=911 span.kind=server system=grpc1921gitlab # [ 258.746610] gitaly[911]: time="2026-11-10T09:54:08.971Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3934 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:08.970" grpc.time_ms=0.578 pid=911 span.kind=server system=grpc1922gitlab # [ 258.765870] gitaly[911]: time="2026-11-10T09:54:08.990Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV diskcache=258074ba-5b17-4302-9eae-2c1e9134e359 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:18.098" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:08.908" pid=911 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1923gitlab # [ 258.775628] gitaly[911]: time="2026-11-10T09:54:09.000Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=66 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5150 command.oublock=8 command.real_time_ms=73 command.spawn_token_fork_ms=0 command.system_time_ms=41 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:18.098" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:08.908" grpc.time_ms=82.373 pid=911 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1924gitlab # [ 258.804289] gitaly[911]: time="2026-11-10T09:54:09.029Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T09:54:19.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:09.028" grpc.time_ms=0.241 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 258.827257] gitaly[911]: time="2026-11-10T09:54:09.052Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=939 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T09:54:19.014" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:09.035" grpc.time_ms=16.416 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926gitlab # [ 258.942996] gitaly[911]: time="2026-11-10T09:54:09.167Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-10T09:54:39.060" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:09.160" grpc.time_ms=7.038 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 258.970149] gitaly[911]: time="2026-11-10T09:54:09.194Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T09:54:19.174" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:09.194" grpc.time_ms=0.197 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 258.977208] gitaly[911]: time="2026-11-10T09:54:09.202Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T09:54:19.176" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T09:54:09.196" grpc.time_ms=5.177 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 258.997835] gitaly[911]: time="2026-11-10T09:54:09.222Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:19.191" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:09.211" grpc.time_ms=10.985 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 259.017988] gitaly[911]: time="2026-11-10T09:54:09.242Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:19.217" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:09.237" grpc.time_ms=5.352 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931gitlab # [ 259.053366] gitaly[911]: time="2026-11-10T09:54:09.278Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=955 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T09:54:39.162" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:09.262" grpc.time_ms=15.455 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a'1933gitlab # [ 259.271520] nixbot[2700]: {"ts": "2026-11-10T09:54:09.495Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47502 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1\" 200"}1934gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', in 0.03 seconds)1935gitlab # [ 259.470512] gitaly[911]: time="2026-11-10T09:54:09.695Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T09:54:19.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:09.694" grpc.time_ms=1.276 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936gitlab # [ 259.503075] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:09.726Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 64ce515318877fcd7b1ffb60d5f22da640a32041 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c15d539b39a6935a66044a4d","created_at":"2026-11-10T09:53:18.131Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-0ad81291b4f34ac1","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:53:18.152Z","job_size_bytes":185,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c15d539b39a6935a66044a4d: done: 0.93802 sec","job_status":"done","queue_duration_s":50.635501,"scheduling_latency_s":50.635531,"gitaly_calls":13,"gitaly_duration_s":0.267735,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.041184,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003764,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005478,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001765,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003493,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.015584,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.0111,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.19,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.217,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.553771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000449778,"exclusive_lock_hold_duration_s":0.028069769,"duration_s":0.93802,"completed_at":"2026-11-10T09:54:09.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.094707,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1937gitlab # [ 259.510283] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:09.728Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"de6318799845a57512475cea","created_at":"2026-11-10T09:53:23.327Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.337Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-de6318799845a57512475cea: start","job_status":"start","queue_duration_s":46.39058,"scheduling_latency_s":46.390611}1938gitlab # [ 259.643549] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:09.867Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"de6318799845a57512475cea","created_at":"2026-11-10T09:53:23.327Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.337Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-de6318799845a57512475cea: done: 0.139507 sec","job_status":"done","queue_duration_s":46.39058,"scheduling_latency_s":46.390611,"redis_calls":16,"redis_duration_s":0.003454,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001986,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000351,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000276292,"duration_s":0.139507,"completed_at":"2026-11-10T09:54:09.867Z","load_balancing_strategy":"primary","db_duration_s":0.015068,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1939gitlab # [ 259.685136] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:09.909Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3b750d1dc9ef2eff6b1bd57e","created_at":"2026-11-10T09:53:23.338Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.345Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3b750d1dc9ef2eff6b1bd57e: start","job_status":"start","queue_duration_s":46.563689,"scheduling_latency_s":46.563727}1940gitlab # [ 259.804849] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.028Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3b750d1dc9ef2eff6b1bd57e","created_at":"2026-11-10T09:53:23.338Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.345Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3b750d1dc9ef2eff6b1bd57e: done: 0.119779 sec","job_status":"done","queue_duration_s":46.563689,"scheduling_latency_s":46.563727,"redis_calls":21,"redis_duration_s":0.0070810000000000005,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003174,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000649524,"duration_s":0.119779,"completed_at":"2026-11-10T09:54:10.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1941gitlab # [ 259.807295] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.031Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a0ddbf0e58f3553db6a1069d","created_at":"2026-11-10T09:53:23.485Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.494Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a0ddbf0e58f3553db6a1069d: start","job_status":"start","queue_duration_s":46.537044,"scheduling_latency_s":46.537089}1942gitlab # [ 260.022526] gitaly[911]: time="2026-11-10T09:54:10.246Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:10.245" grpc.time_ms=1.051 pid=911 span.kind=server system=grpc1943gitlab # [ 260.074919] gitaly[911]: time="2026-11-10T09:54:10.299Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:10.299" grpc.time_ms=0.393 pid=911 span.kind=server system=grpc1944gitlab # [ 260.118415] gitaly[911]: time="2026-11-10T09:54:10.343Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:10.342" grpc.time_ms=0.607 pid=911 span.kind=server system=grpc1945gitlab # [ 260.143869] gitaly[911]: time="2026-11-10T09:54:10.368Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE diskcache=3ffbc621-c6ac-4fe1-8b4b-0128de31b3dc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T09:54:40.116" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:10.216" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946gitlab # [ 260.157994] gitaly[911]: time="2026-11-10T09:54:10.382Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=115 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7247 command.oublock=0 command.real_time_ms=144 command.spawn_token_fork_ms=0 command.system_time_ms=78 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T09:54:40.116" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:10.216" grpc.time_ms=152.076 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a'1948gitlab # [ 260.293451] gitaly[911]: time="2026-11-10T09:54:10.518Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T09:54:40.416" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.517" grpc.time_ms=0.98 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949gitlab # [ 260.322722] gitaly[911]: time="2026-11-10T09:54:10.547Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0SCAPN7VK5E8JEC7VDXE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:40.446" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.546" grpc.time_ms=0.816 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950gitlab # [ 260.335972] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.560Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a0ddbf0e58f3553db6a1069d","created_at":"2026-11-10T09:53:23.485Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.494Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2576,"job_status":"rescheduled","message":"PipelineProcessWorker JID-a0ddbf0e58f3553db6a1069d: rescheduled"}1951gitlab # [ 260.348349] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.572Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a0ddbf0e58f3553db6a1069d","created_at":"2026-11-10T09:53:23.485Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-b3c82ae9fb654ee6","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:53:23.494Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a0ddbf0e58f3553db6a1069d: done: 0.540995 sec","job_status":"done","queue_duration_s":46.537044,"scheduling_latency_s":46.537089,"gitaly_calls":3,"gitaly_duration_s":0.174428,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.02862,"redis_read_bytes":460,"redis_write_bytes":29294,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001071,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001268,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009674,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004651,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15581,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.007295,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000371,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000726,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002493,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.275113,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.46989994599999996,"duration_s":0.540995,"completed_at":"2026-11-10T09:54:10.572Z","load_balancing_strategy":"primary","db_duration_s":0.068269,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1952gitlab # [ 260.389982] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.574Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","64ce515318877fcd7b1ffb60d5f22da640a32041 a59288aee385d50e3b1a10cc2d2c3866b2c77c0a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2658ebc5fcd8845179204883","created_at":"2026-11-10T09:54:08.180Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-032d04fa947946f9","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:08.184Z","job_size_bytes":185,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2658ebc5fcd8845179204883: start","job_status":"start","queue_duration_s":2.390594,"scheduling_latency_s":2.390622}1953gitlab # [ 260.390976] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.587Z","message":"changes: \"64ce515318877fcd7b1ffb60d5f22da640a32041 a59288aee385d50e3b1a10cc2d2c3866b2c77c0a refs/heads/master\\n\""}1954gitlab # [ 260.392410] gitaly[911]: time="2026-11-10T09:54:10.617Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:20.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:10.606" grpc.time_ms=10.592 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1955gitlab # [ 260.404676] gitaly[911]: time="2026-11-10T09:54:10.629Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T09:54:20.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:10.621" grpc.time_ms=7.877 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1956gitlab # [ 260.411061] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.632Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"74c5dfa97758838f240168aa","created_at":"2026-11-10T09:54:10.631Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"cc8a20c3dd41dbb400ab6efb","job_size_bytes":46,"pid":2576,"job_status":"deduplicated","message":"ProjectCacheWorker JID-74c5dfa97758838f240168aa: deduplicated: until executing","deduplication.type":"until executing"}1957gitlab # [ 260.456824] gitaly[911]: time="2026-11-10T09:54:10.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T09:54:20.044" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:10.644" grpc.time_ms=5.73 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958gitlab # [ 260.457632] gitaly[911]: time="2026-11-10T09:54:10.662Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-10T09:54:40.553" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.653" grpc.time_ms=8.58 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1959gitlab # [ 260.458226] gitaly[911]: time="2026-11-10T09:54:10.672Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T09:54:20.068" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T09:54:10.668" grpc.time_ms=3.364 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 260.460102] gitaly[911]: time="2026-11-10T09:54:10.683Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:20.077" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:10.677" grpc.time_ms=5.406 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 260.525108] nixbot[2700]: {"ts": "2026-11-10T09:54:10.749Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47506 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1\" 200"}1962gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', in 0.25 seconds)1963gitlab # [ 260.533981] gitaly[911]: time="2026-11-10T09:54:10.758Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T09:54:20.067" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.758" grpc.time_ms=0.812 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964gitlab # [ 260.547837] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.772Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","64ce515318877fcd7b1ffb60d5f22da640a32041 a59288aee385d50e3b1a10cc2d2c3866b2c77c0a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2658ebc5fcd8845179204883","created_at":"2026-11-10T09:54:08.180Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-032d04fa947946f9","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:08.184Z","job_size_bytes":185,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2658ebc5fcd8845179204883: done: 0.197458 sec","job_status":"done","queue_duration_s":2.390594,"scheduling_latency_s":2.390622,"gitaly_calls":7,"gitaly_duration_s":0.054633,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.018563,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.001738,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000993,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002678,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006948,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006206,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.021,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.115124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000713219,"exclusive_lock_hold_duration_s":0.021095139,"duration_s":0.197458,"completed_at":"2026-11-10T09:54:10.772Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018952,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1965gitlab # [ 260.551918] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.773Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"cc8a20c3dd41dbb400ab6efb","created_at":"2026-11-10T09:54:08.850Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:08.869Z","job_size_bytes":46,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-cc8a20c3dd41dbb400ab6efb: start","job_status":"start","queue_duration_s":1.903704,"scheduling_latency_s":1.903724}1966gitlab # [ 260.595292] gitaly[911]: time="2026-11-10T09:54:10.806Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-10T15:54:10.803" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:10.804" grpc.time_ms=2.182 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 260.596273] gitaly[911]: time="2026-11-10T09:54:10.821Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T09:54:40.714" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.814" grpc.time_ms=6.807 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 260.617174] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.841Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"cc8a20c3dd41dbb400ab6efb","created_at":"2026-11-10T09:54:08.850Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:08.869Z","job_size_bytes":46,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-cc8a20c3dd41dbb400ab6efb: done: 0.068445 sec","job_status":"done","queue_duration_s":1.903704,"scheduling_latency_s":1.903724,"gitaly_calls":2,"gitaly_duration_s":0.012324,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.005516,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001351,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000438,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00194,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000237181,"duration_s":0.068445,"completed_at":"2026-11-10T09:54:10.841Z","load_balancing_strategy":"primary","db_duration_s":0.010745,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1969gitlab # [ 260.663159] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.843Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"9c7c523a0c7bd2437fc47169","created_at":"2026-11-10T09:54:08.883Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:08.887Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-9c7c523a0c7bd2437fc47169: start","job_status":"start","queue_duration_s":1.955555,"scheduling_latency_s":1.955571}1970gitlab # [ 260.663931] gitaly[911]: time="2026-11-10T09:54:10.879Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV diskcache=a8137f5f-7973-49ff-8240-5d3203e61b65 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T15:54:10.858" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.859" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971gitlab # [ 260.664542] gitaly[911]: time="2026-11-10T09:54:10.879Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1284 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T15:54:10.858" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.859" grpc.time_ms=20.025 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 260.683207] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.907Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"9c7c523a0c7bd2437fc47169","created_at":"2026-11-10T09:54:08.883Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:08.887Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-9c7c523a0c7bd2437fc47169: done: 0.064927 sec","job_status":"done","queue_duration_s":1.955555,"scheduling_latency_s":1.955571,"gitaly_calls":1,"gitaly_duration_s":0.02196,"redis_calls":15,"redis_duration_s":0.002695,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000321,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000465,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.048549467,"duration_s":0.064927,"completed_at":"2026-11-10T09:54:10.907Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013937,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 260.723747] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.909Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"6c3b713cee7650affd4195ec","created_at":"2026-11-10T09:54:09.023Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:09.024Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6c3b713cee7650affd4195ec: start","job_status":"start","queue_duration_s":1.884185,"scheduling_latency_s":1.8842}1974gitlab # [ 260.724554] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.941Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"6c3b713cee7650affd4195ec","created_at":"2026-11-10T09:54:09.023Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:09.024Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6c3b713cee7650affd4195ec: done: 0.032774 sec","job_status":"done","queue_duration_s":1.884185,"scheduling_latency_s":1.8842,"gitaly_calls":1,"gitaly_duration_s":0.007087,"redis_calls":16,"redis_duration_s":0.002925,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000251,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001212,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000579,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032774,"completed_at":"2026-11-10T09:54:10.941Z","load_balancing_strategy":"primary","db_duration_s":0.002203,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1975gitlab # [ 260.777290] gitaly[911]: time="2026-11-10T09:54:10.932Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T09:54:40.826" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.926" grpc.time_ms=5.313 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 260.777927] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:10.943Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"89e27c5ff1d23d2d3c7d86f0","created_at":"2026-11-10T09:54:09.540Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bb0c8e7bbf7fd80f44ba873b7ed61c4b2d7df49b1bc8f3f7bbe468bd8f2fe47","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:09.546Z","job_size_bytes":467,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-89e27c5ff1d23d2d3c7d86f0: start","job_status":"start","queue_duration_s":1.396666,"scheduling_latency_s":1.396694}1977gitlab # [ 260.781242] gitaly[911]: time="2026-11-10T09:54:10.970Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:40.869" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:10.969" grpc.time_ms=0.499 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978gitlab # [ 260.819266] gitaly[911]: time="2026-11-10T09:54:10.977Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T09:54:20.001" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:10.971" grpc.time_ms=5.739 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979gitlab # [ 260.832245] gitaly[911]: time="2026-11-10T09:54:11.057Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T09:54:41.056" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:11.056" grpc.time_ms=0.605 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 260.836303] gitaly[911]: time="2026-11-10T09:54:11.061Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=50 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T09:54:41.059" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:11.059" grpc.time_ms=1.639 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981gitlab # [ 260.902675] gitaly[911]: time="2026-11-10T09:54:11.127Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:21.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:11.121" grpc.time_ms=6.209 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982gitlab # [ 261.369399] gitaly[911]: time="2026-11-10T09:54:11.593Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T09:54:41.576" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:11.576" grpc.time_ms=16.448 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 261.386707] gitaly[911]: time="2026-11-10T09:54:11.611Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E0M8BFKTXF3JYKDHTP2JV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T09:54:41.595" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:11.595" grpc.time_ms=15.603 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 261.450707] sidekiq-cluster[2576]: {"severity":"WARN","time":"2026-11-10T09:54:11.675Z","class":"CreatePipelineWorker","correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"64ce515318877fcd7b1ffb60d5f22da640a32041","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"64ce515318877fcd7b1ffb60d5f22da640a32041","ref":"refs/heads/master"},"retry":0}1985gitlab # [ 261.456179] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.680Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"89e27c5ff1d23d2d3c7d86f0","created_at":"2026-11-10T09:54:09.540Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bb0c8e7bbf7fd80f44ba873b7ed61c4b2d7df49b1bc8f3f7bbe468bd8f2fe47","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:09.546Z","job_size_bytes":467,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-89e27c5ff1d23d2d3c7d86f0: done: 0.737416 sec","job_status":"done","queue_duration_s":1.396666,"scheduling_latency_s":1.396694,"gitaly_calls":7,"gitaly_duration_s":0.099841,"redis_calls":51,"redis_duration_s":0.010648,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000566,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003833,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003974,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.579137,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000309536,"duration_s":0.737416,"completed_at":"2026-11-10T09:54:11.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045154,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1986gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a'1987gitlab # [ 261.474500] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.682Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"94457011-b27a-4d71-9670-2d0365178d92\"}"],"class":"WebHookWorker","jid":"8f9101c30fe791a100e6a925","created_at":"2026-11-10T09:54:09.661Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fa75fc9e1fc41b095581eafc424aca9111260a74a2219435c6737844b52b64b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:09.669Z","job_size_bytes":1751,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8f9101c30fe791a100e6a925: start","job_status":"start","queue_duration_s":2.013268,"scheduling_latency_s":2.013285}1988gitlab # [ 261.575613] nixbot[2700]: {"ts": "2026-11-10T09:54:11.799Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "7d406f09-524c-4df8-ae75-0c121adcd661", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "64ce515318877fcd7b1ffb60d5f22da640a32041", "pr": null}1989gitlab # [ 261.578499] nixbot[2700]: {"ts": "2026-11-10T09:54:11.803Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47512 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1\" 200"}1990gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', in 0.05 seconds)1991gitlab # [ 261.586586] nixbot[2700]: {"ts": "2026-11-10T09:54:11.811Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47510 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1992gitlab # [ 261.610984] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.834Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"94457011-b27a-4d71-9670-2d0365178d92\"}"],"class":"WebHookWorker","jid":"8f9101c30fe791a100e6a925","created_at":"2026-11-10T09:54:09.661Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f380b6de8214b99","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fa75fc9e1fc41b095581eafc424aca9111260a74a2219435c6737844b52b64b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:09.669Z","job_size_bytes":1751,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8f9101c30fe791a100e6a925: done: 0.152415 sec","job_status":"done","queue_duration_s":2.013268,"scheduling_latency_s":2.013285,"redis_calls":19,"redis_duration_s":0.004807,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001062,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000527,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002189,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.045726203000015175,"cpu_s":0.088781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000541968,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.152415,"completed_at":"2026-11-10T09:54:11.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1993gitlab # [ 261.632349] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.837Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"64af79bb0709ebe7fef89f77","created_at":"2026-11-10T09:54:10.205Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.208Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-64af79bb0709ebe7fef89f77: start","job_status":"start","queue_duration_s":1.628308,"scheduling_latency_s":1.628343}1994gitlab # [ 261.677934] gitaly[911]: time="2026-11-10T09:54:11.902Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E297RXDZ939QPRR83A53W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:11.894" grpc.time_ms=8.299 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 261.684801] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.909Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"64af79bb0709ebe7fef89f77","created_at":"2026-11-10T09:54:10.205Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.208Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-64af79bb0709ebe7fef89f77: done: 0.072278 sec","job_status":"done","queue_duration_s":1.628308,"scheduling_latency_s":1.628343,"redis_calls":10,"redis_duration_s":0.002038,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000397,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218463,"duration_s":0.072278,"completed_at":"2026-11-10T09:54:11.909Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015543,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1996gitlab # [ 261.725226] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.910Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b37bceaf3497d0028cbe5269","created_at":"2026-11-10T09:54:10.389Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.396Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b37bceaf3497d0028cbe5269: start","job_status":"start","queue_duration_s":1.513813,"scheduling_latency_s":1.51383}1997gitlab # [ 261.725978] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.931Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"b37bceaf3497d0028cbe5269","created_at":"2026-11-10T09:54:10.389Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.396Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-b37bceaf3497d0028cbe5269: done: 0.021315 sec","job_status":"done","queue_duration_s":1.513813,"scheduling_latency_s":1.51383,"redis_calls":10,"redis_duration_s":0.0012109999999999998,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000743,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199467,"duration_s":0.021315,"completed_at":"2026-11-10T09:54:11.931Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1998gitlab # [ 261.771238] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.932Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d99677320eae505741a88450","created_at":"2026-11-10T09:54:10.412Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.418Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d99677320eae505741a88450: start","job_status":"start","queue_duration_s":1.514254,"scheduling_latency_s":1.51427}1999gitlab # [ 261.772067] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.953Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d99677320eae505741a88450","created_at":"2026-11-10T09:54:10.412Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.418Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d99677320eae505741a88450: done: 0.021036 sec","job_status":"done","queue_duration_s":1.514254,"scheduling_latency_s":1.51427,"redis_calls":10,"redis_duration_s":0.001205,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.021036,"completed_at":"2026-11-10T09:54:11.953Z","load_balancing_strategy":"primary","db_duration_s":0.00306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2000gitlab # [ 261.821885] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.954Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4ad5abb1d9543e65c3c0fbfa","created_at":"2026-11-10T09:54:10.435Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.441Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4ad5abb1d9543e65c3c0fbfa: start","job_status":"start","queue_duration_s":1.513138,"scheduling_latency_s":1.513153}2001gitlab # [ 261.822753] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.977Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4ad5abb1d9543e65c3c0fbfa","created_at":"2026-11-10T09:54:10.435Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.441Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4ad5abb1d9543e65c3c0fbfa: done: 0.022997 sec","job_status":"done","queue_duration_s":1.513138,"scheduling_latency_s":1.513153,"redis_calls":13,"redis_duration_s":0.001534,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000355,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217346,"duration_s":0.022997,"completed_at":"2026-11-10T09:54:11.977Z","load_balancing_strategy":"primary","db_duration_s":0.003075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002gitlab # [ 261.870561] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.978Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e7bd5ceed2c14fe2f9dd5171","created_at":"2026-11-10T09:54:10.442Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.447Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e7bd5ceed2c14fe2f9dd5171: start","job_status":"start","queue_duration_s":1.530904,"scheduling_latency_s":1.530919}2003gitlab # [ 261.873130] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e7bd5ceed2c14fe2f9dd5171","created_at":"2026-11-10T09:54:10.442Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.447Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e7bd5ceed2c14fe2f9dd5171: done: 0.019817 sec","job_status":"done","queue_duration_s":1.530904,"scheduling_latency_s":1.530919,"redis_calls":10,"redis_duration_s":0.001204,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203937,"duration_s":0.019817,"completed_at":"2026-11-10T09:54:11.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 261.930843] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:11.999Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d6c2378be60e4f0ba2b9df3a","created_at":"2026-11-10T09:54:10.448Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.454Z","job_size_bytes":19,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d6c2378be60e4f0ba2b9df3a: start","job_status":"start","queue_duration_s":1.544946,"scheduling_latency_s":1.54496}2005gitlab # [ 261.931657] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.018Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d6c2378be60e4f0ba2b9df3a","created_at":"2026-11-10T09:54:10.448Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.454Z","job_size_bytes":19,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d6c2378be60e4f0ba2b9df3a: done: 0.018925 sec","job_status":"done","queue_duration_s":1.544946,"scheduling_latency_s":1.54496,"redis_calls":11,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000222,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000217,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000211479,"duration_s":0.018925,"completed_at":"2026-11-10T09:54:12.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002472,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006gitlab # [ 261.978709] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.019Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"21b2f2e56726d0ead81b09d8","created_at":"2026-11-10T09:54:10.458Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.464Z","job_size_bytes":66,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-21b2f2e56726d0ead81b09d8: start","job_status":"start","queue_duration_s":1.554651,"scheduling_latency_s":1.554666}2007gitlab # [ 261.979569] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.086Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"21b2f2e56726d0ead81b09d8","created_at":"2026-11-10T09:54:10.458Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.464Z","job_size_bytes":66,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-21b2f2e56726d0ead81b09d8: done: 0.067414 sec","job_status":"done","queue_duration_s":1.554651,"scheduling_latency_s":1.554666,"redis_calls":10,"redis_duration_s":0.0012929999999999999,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00031,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212318,"duration_s":0.067414,"completed_at":"2026-11-10T09:54:12.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 262.034086] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.087Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4eb9d5f04b0c553d33326475","created_at":"2026-11-10T09:54:10.465Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.471Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4eb9d5f04b0c553d33326475: start","job_status":"start","queue_duration_s":1.61645,"scheduling_latency_s":1.616466}2009gitlab # [ 262.034817] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.112Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4eb9d5f04b0c553d33326475","created_at":"2026-11-10T09:54:10.465Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.471Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4eb9d5f04b0c553d33326475: done: 0.024828 sec","job_status":"done","queue_duration_s":1.61645,"scheduling_latency_s":1.616466,"redis_calls":10,"redis_duration_s":0.001259,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200025,"duration_s":0.024828,"completed_at":"2026-11-10T09:54:12.112Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 262.076091] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.113Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2899fcbb720102bb8fd703e9","created_at":"2026-11-10T09:54:10.478Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.483Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2899fcbb720102bb8fd703e9: start","job_status":"start","queue_duration_s":1.629916,"scheduling_latency_s":1.629931}2011gitlab # [ 262.126697] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.155Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2899fcbb720102bb8fd703e9","created_at":"2026-11-10T09:54:10.478Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.483Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2899fcbb720102bb8fd703e9: done: 0.041533 sec","job_status":"done","queue_duration_s":1.629916,"scheduling_latency_s":1.629931,"redis_calls":10,"redis_duration_s":0.0016640000000000001,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000297,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362616,"duration_s":0.041533,"completed_at":"2026-11-10T09:54:12.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2012gitlab # [ 262.130792] gitaly[911]: time="2026-11-10T09:54:12.277Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z diskcache=451d7dd9-b6d5-457d-93c8-1d991cf42a90 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T15:54:12.267" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.268" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2013gitlab # [ 262.175400] gitaly[911]: time="2026-11-10T09:54:12.277Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T15:54:12.267" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.268" grpc.time_ms=9.172 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2014gitlab # [ 262.176121] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.156Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"10d06ffd472465345ac78a62","created_at":"2026-11-10T09:54:10.484Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.488Z","job_size_bytes":6,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-10d06ffd472465345ac78a62: start","job_status":"start","queue_duration_s":1.667729,"scheduling_latency_s":1.667745}2015gitlab # [ 262.176725] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.171Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"10d06ffd472465345ac78a62","created_at":"2026-11-10T09:54:10.484Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.488Z","job_size_bytes":6,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-10d06ffd472465345ac78a62: done: 0.014937 sec","job_status":"done","queue_duration_s":1.667729,"scheduling_latency_s":1.667745,"redis_calls":10,"redis_duration_s":0.001357,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214273,"duration_s":0.014937,"completed_at":"2026-11-10T09:54:12.171Z","load_balancing_strategy":"primary","db_duration_s":0.001442,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016gitlab # [ 262.218916] gitaly[911]: time="2026-11-10T09:54:12.303Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:42.301" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.302" grpc.time_ms=1.036 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2017gitlab # [ 262.220149] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"01e40839800a8e75ac8e31d9","created_at":"2026-11-10T09:54:10.489Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.493Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-01e40839800a8e75ac8e31d9: start","job_status":"start","queue_duration_s":1.67884,"scheduling_latency_s":1.678854}2018gitlab # [ 262.265815] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.195Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"01e40839800a8e75ac8e31d9","created_at":"2026-11-10T09:54:10.489Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.493Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-01e40839800a8e75ac8e31d9: done: 0.023715 sec","job_status":"done","queue_duration_s":1.67884,"scheduling_latency_s":1.678854,"redis_calls":10,"redis_duration_s":0.0015210000000000002,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000557334,"duration_s":0.023715,"completed_at":"2026-11-10T09:54:12.195Z","load_balancing_strategy":"primary","db_duration_s":0.003994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 262.305184] gitaly[911]: time="2026-11-10T09:54:12.315Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T09:54:22.185" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:12.305" grpc.time_ms=9.577 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2020gitlab # [ 262.306138] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.197Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"278689c6eee408582cdb0207","created_at":"2026-11-10T09:54:10.499Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.504Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-278689c6eee408582cdb0207: start","job_status":"start","queue_duration_s":1.692239,"scheduling_latency_s":1.692257}2021gitlab # [ 262.307241] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.238Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"278689c6eee408582cdb0207","created_at":"2026-11-10T09:54:10.499Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.504Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-278689c6eee408582cdb0207: done: 0.041533 sec","job_status":"done","queue_duration_s":1.692239,"scheduling_latency_s":1.692257,"redis_calls":14,"redis_duration_s":0.003056,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000477994,"duration_s":0.041533,"completed_at":"2026-11-10T09:54:12.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022gitlab # [ 262.366445] gitaly[911]: time="2026-11-10T09:54:12.339Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T09:54:42.339" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.339" grpc.time_ms=0.633 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2023gitlab # [ 262.367200] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.239Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a459b591b5b18ed621206919","created_at":"2026-11-10T09:54:10.561Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.563Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a459b591b5b18ed621206919: start","job_status":"start","queue_duration_s":1.676185,"scheduling_latency_s":1.6762}2024gitlab # [ 262.367815] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.255Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a459b591b5b18ed621206919","created_at":"2026-11-10T09:54:10.561Z","trace_propagation_headers":{"sentry-trace":"b580cf5406c5402c8e86ac655b22f20f-06aaaf6a519e43fb","baggage":"sentry-trace_id=b580cf5406c5402c8e86ac655b22f20f,sentry-release=Unknown"},"correlation_id":"01M72E0SCAPN7VK5E8JEC7VDXE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.563Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a459b591b5b18ed621206919: done: 0.016355 sec","job_status":"done","queue_duration_s":1.676185,"scheduling_latency_s":1.6762,"redis_calls":8,"redis_duration_s":0.0010479999999999999,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016355,"completed_at":"2026-11-10T09:54:12.255Z","load_balancing_strategy":"primary","db_duration_s":0.002307,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2025gitlab # [ 262.406435] gitaly[911]: time="2026-11-10T09:54:12.343Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=38 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T09:54:42.342" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.342" grpc.time_ms=1.577 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2026gitlab # [ 262.409793] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.256Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0d742b5f59bbb0f87e1d6486","created_at":"2026-11-10T09:54:10.639Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"9c7c523a0c7bd2437fc47169","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.641Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0d742b5f59bbb0f87e1d6486: start","job_status":"start","queue_duration_s":1.615424,"scheduling_latency_s":1.615441}2027gitlab # [ 262.455545] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.287Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0d742b5f59bbb0f87e1d6486","created_at":"2026-11-10T09:54:10.639Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"9c7c523a0c7bd2437fc47169","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.641Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0d742b5f59bbb0f87e1d6486: done: 0.030147 sec","job_status":"done","queue_duration_s":1.615424,"scheduling_latency_s":1.615441,"gitaly_calls":1,"gitaly_duration_s":0.01099,"redis_calls":12,"redis_duration_s":0.001502,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000326,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.019153552,"duration_s":0.030147,"completed_at":"2026-11-10T09:54:12.286Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 262.466691] gitaly[911]: time="2026-11-10T09:54:12.433Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:22.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:12.426" grpc.time_ms=6.801 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029gitlab # [ 262.478867] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.288Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0b69f431c8b32e54993ed9d4","created_at":"2026-11-10T09:54:10.732Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b4745e9402c1865292c317aa42e2d7edd4a5a2a706c344247381a5b9d410f69","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.734Z","job_size_bytes":467,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0b69f431c8b32e54993ed9d4: start","job_status":"start","queue_duration_s":1.553499,"scheduling_latency_s":1.553515}2030gitlab # [ 262.536436] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 9282031gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a'2032gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', in 0.03 seconds)2033gitlab # [ 262.609573] nixbot[2700]: {"ts": "2026-11-10T09:54:12.833Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47516 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1\" 200"}2034gitlab # [ 262.632816] gitaly[911]: time="2026-11-10T09:54:12.857Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2A4WRHFPDWY404FP44H8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:12.848" pid=911 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2035gitlab # [ 262.633571] gitaly[911]: time="2026-11-10T09:54:12.857Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2A4WRHFPDWY404FP44H8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:12.848" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2036gitlab # [ 262.635095] gitaly[911]: time="2026-11-10T09:54:12.859Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2A4WRHFPDWY404FP44H8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:12.848" grpc.time_ms=11.467 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2037gitlab # [ 262.636181] gitlab-workhorse[825]: time="2026-11-10T09:54:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672038gitlab # [ 262.638133] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 962039gitlab # [ 262.687356] gitaly[911]: time="2026-11-10T09:54:12.912Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=957 command.oublock=0 command.real_time_ms=47 command.spawn_token_fork_ms=40 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T09:54:42.861" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.861" grpc.time_ms=50.736 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 262.702140] gitaly[911]: time="2026-11-10T09:54:12.926Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E255ZJ2VBBPT028CKWW3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T09:54:42.015" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:12.915" grpc.time_ms=10.109 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 262.715590] sidekiq-cluster[2576]: {"severity":"WARN","time":"2026-11-10T09:54:12.939Z","class":"CreatePipelineWorker","correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"64ce515318877fcd7b1ffb60d5f22da640a32041","after":"a59288aee385d50e3b1a10cc2d2c3866b2c77c0a","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a59288aee385d50e3b1a10cc2d2c3866b2c77c0a","ref":"refs/heads/master"},"retry":0}2042gitlab # [ 262.719324] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.943Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0b69f431c8b32e54993ed9d4","created_at":"2026-11-10T09:54:10.732Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b4745e9402c1865292c317aa42e2d7edd4a5a2a706c344247381a5b9d410f69","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.734Z","job_size_bytes":467,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0b69f431c8b32e54993ed9d4: done: 0.655924 sec","job_status":"done","queue_duration_s":1.553499,"scheduling_latency_s":1.553515,"gitaly_calls":7,"gitaly_duration_s":0.15398,"redis_calls":33,"redis_duration_s":0.007513000000000001,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000701,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002941,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003125,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.537302,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.655924,"completed_at":"2026-11-10T09:54:12.943Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014868,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2043gitlab # [ 262.725072] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.945Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"995efefb-6647-40cc-89b5-05f3b1d972b9\"}"],"class":"WebHookWorker","jid":"75eb65bb43faaa7432c821d8","created_at":"2026-11-10T09:54:10.747Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23a7bbd61084d46773bff1ceefc8f58bf712c06fb9d177cc44aadedb793a42c4","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.748Z","job_size_bytes":1749,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-75eb65bb43faaa7432c821d8: start","job_status":"start","queue_duration_s":2.19659,"scheduling_latency_s":2.19661}2044gitlab # [ 262.741078] nixbot[2700]: {"ts": "2026-11-10T09:54:12.963Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "126c002c-b2fd-4404-8aeb-b785e33dd20c", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "a59288aee385d50e3b1a10cc2d2c3866b2c77c0a", "pr": null}2045gitlab # [ 262.744875] nixbot[2700]: {"ts": "2026-11-10T09:54:12.966Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47528 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}2046gitlab # [ 262.748728] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.973Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"995efefb-6647-40cc-89b5-05f3b1d972b9\"}"],"class":"WebHookWorker","jid":"75eb65bb43faaa7432c821d8","created_at":"2026-11-10T09:54:10.747Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-a51a0c1c0d274b5a","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23a7bbd61084d46773bff1ceefc8f58bf712c06fb9d177cc44aadedb793a42c4","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.748Z","job_size_bytes":1749,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-75eb65bb43faaa7432c821d8: done: 0.028226 sec","job_status":"done","queue_duration_s":2.19659,"scheduling_latency_s":2.19661,"redis_calls":14,"redis_duration_s":0.002105,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000213,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.005654629999980898,"cpu_s":0.016404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.028226,"completed_at":"2026-11-10T09:54:12.973Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 262.805048] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:12.974Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"862736f705f1f80cb60820c8","created_at":"2026-11-10T09:54:10.830Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-3ac1e74164c449fb","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.833Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-862736f705f1f80cb60820c8: start","job_status":"start","queue_duration_s":2.14099,"scheduling_latency_s":2.141006}2048gitlab # [ 262.805310] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.005Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"862736f705f1f80cb60820c8","created_at":"2026-11-10T09:54:10.830Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-3ac1e74164c449fb","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:10.833Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-862736f705f1f80cb60820c8: done: 0.030592 sec","job_status":"done","queue_duration_s":2.14099,"scheduling_latency_s":2.141006,"redis_calls":19,"redis_duration_s":0.002462,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000354,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004773791,"duration_s":0.030592,"completed_at":"2026-11-10T09:54:13.005Z","load_balancing_strategy":"primary","db_duration_s":0.00514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049gitlab # [ 262.841769] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.006Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8f9101c30fe791a100e6a925"],"class":"WebHooks::LogExecutionWorker","jid":"fda8407a6e55df4808149dce","created_at":"2026-11-10T09:54:11.814Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-aa859e23867a4d16","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b28e73ec57c365e1786a03620ea0e3c7190ce0fe0d674efd7f873b413d2f4940","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:11.822Z","job_size_bytes":2593,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-fda8407a6e55df4808149dce: start","job_status":"start","queue_duration_s":1.183147,"scheduling_latency_s":1.183192}2050gitlab # [ 262.889030] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.035Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8f9101c30fe791a100e6a925"],"class":"WebHooks::LogExecutionWorker","jid":"fda8407a6e55df4808149dce","created_at":"2026-11-10T09:54:11.814Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-aa859e23867a4d16","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b28e73ec57c365e1786a03620ea0e3c7190ce0fe0d674efd7f873b413d2f4940","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:11.822Z","job_size_bytes":2593,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-fda8407a6e55df4808149dce: done: 0.029119 sec","job_status":"done","queue_duration_s":1.183147,"scheduling_latency_s":1.183192,"redis_calls":10,"redis_duration_s":0.001409,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022461,"duration_s":0.029119,"completed_at":"2026-11-10T09:54:13.035Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051gitlab # [ 262.896035] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.036Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","75eb65bb43faaa7432c821d8"],"class":"WebHooks::LogExecutionWorker","jid":"003b13e950f1cc53799dd006","created_at":"2026-11-10T09:54:12.967Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-eddd7b3872c14c59","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6131c8527b0c203ae947dc3ad25ca780e76add0e64573a5a76bbfd9525dd0e45","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:12.969Z","job_size_bytes":2592,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-003b13e950f1cc53799dd006: start","job_status":"start","queue_duration_s":0.066985,"scheduling_latency_s":0.066999}2052gitlab # [ 262.946116] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.051Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","75eb65bb43faaa7432c821d8"],"class":"WebHooks::LogExecutionWorker","jid":"003b13e950f1cc53799dd006","created_at":"2026-11-10T09:54:12.967Z","trace_propagation_headers":{"sentry-trace":"ad73c0bd13bc4d8eb135cf97d363a015-eddd7b3872c14c59","baggage":"sentry-trace_id=ad73c0bd13bc4d8eb135cf97d363a015,sentry-release=Unknown"},"correlation_id":"01M72E255ZJ2VBBPT028CKWW3Z","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6131c8527b0c203ae947dc3ad25ca780e76add0e64573a5a76bbfd9525dd0e45","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:12.969Z","job_size_bytes":2592,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-003b13e950f1cc53799dd006: done: 0.015671 sec","job_status":"done","queue_duration_s":0.066985,"scheduling_latency_s":0.066999,"redis_calls":7,"redis_duration_s":0.00096,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015671,"completed_at":"2026-11-10T09:54:13.051Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053gitlab # [ 262.986171] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"88aa05d047540755e55a65ce","created_at":"2026-11-10T09:54:12.996Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f0ac210d26b40ba","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:12.999Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-88aa05d047540755e55a65ce: start","job_status":"start","queue_duration_s":0.05301,"scheduling_latency_s":0.053024}2054gitlab # [ 262.986800] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"88aa05d047540755e55a65ce","created_at":"2026-11-10T09:54:12.996Z","trace_propagation_headers":{"sentry-trace":"607e98848f4242f4a7e0ad095e9471ab-9f0ac210d26b40ba","baggage":"sentry-trace_id=607e98848f4242f4a7e0ad095e9471ab,sentry-release=Unknown"},"correlation_id":"01M72E0M8BFKTXF3JYKDHTP2JV","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:12.999Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-88aa05d047540755e55a65ce: done: 0.054735 sec","job_status":"done","queue_duration_s":0.05301,"scheduling_latency_s":0.053024,"redis_calls":11,"redis_duration_s":0.001541,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021539,"duration_s":0.054735,"completed_at":"2026-11-10T09:54:13.107Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055gitlab # [ 263.068813] gitaly[911]: time="2026-11-10T09:54:13.293Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2A982FJ4XXAXV2XQ3X9V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.282" pid=911 remote_ip="2001:db8:1::1" request_sha=d0bdd4d5ecdbc7843f098dbaabc3d9b8e10cfdd5 response_bytes=337 span.kind=server system=grpc2056gitlab # [ 263.069576] gitaly[911]: time="2026-11-10T09:54:13.293Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2A982FJ4XXAXV2XQ3X9V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.282" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2057gitlab # [ 263.070501] gitaly[911]: time="2026-11-10T09:54:13.294Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2A982FJ4XXAXV2XQ3X9V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.282" grpc.time_ms=11.716 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2058gitlab # [ 263.104405] gitlab-workhorse[825]: time="2026-11-10T09:54:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3372059gitlab # [ 263.105130] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 4242060gitlab # [ 263.138809] nixbot[2700]: {"ts": "2026-11-10T09:54:13.363Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2061gitlab # [ 263.143623] nixbot[2700]: {"ts": "2026-11-10T09:54:13.368Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "64ce515318877fcd7b1ffb60d5f22da640a32041"}2062gitlab # [ 263.192367] gitaly[911]: time="2026-11-10T09:54:13.417Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2AQR3WS9JEH8SKBT2ZA4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.410" grpc.time_ms=6.967 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2063gitlab # [ 263.193883] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 492064gitlab # [ 263.237868] gitaly[911]: time="2026-11-10T09:54:13.462Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2ASC0X8E00DQX1SEZ7TJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.452" pid=911 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2065gitlab # [ 263.238981] gitaly[911]: time="2026-11-10T09:54:13.462Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2ASC0X8E00DQX1SEZ7TJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.452" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2066gitlab # [ 263.262692] gitaly[911]: time="2026-11-10T09:54:13.487Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2ASC0X8E00DQX1SEZ7TJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.452" grpc.time_ms=34.74 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2067gitlab # [ 263.263699] gitlab-workhorse[825]: time="2026-11-10T09:54:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672068gitlab # [ 263.264775] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 672069gitlab # [ 263.270859] gitaly[911]: time="2026-11-10T09:54:13.495Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:43.394" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:13.495" grpc.time_ms=0.555 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2070gitlab # [ 263.339383] gitaly[911]: time="2026-11-10T09:54:13.564Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:23.067" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.557" grpc.time_ms=6.518 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab # [ 263.365618] gitaly[911]: time="2026-11-10T09:54:13.590Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72E2AXB7YW0QFSCYJ2PVZS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.583" grpc.time_ms=6.598 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2072gitlab # [ 263.367283] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 432073gitlab # [ 263.383654] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.608Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8eadb050b69cb845ec61f9cd","created_at":"2026-11-10T09:54:13.602Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-26c000337a5f45a7","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.606Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8eadb050b69cb845ec61f9cd: start","job_status":"start","queue_duration_s":0.001855,"scheduling_latency_s":0.001891}2074gitlab # [ 263.399074] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2352075gitlab # [ 263.437385] gitaly[911]: time="2026-11-10T09:54:13.661Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AZ6C1Z5DS898403TPP9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.649" pid=911 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2076gitlab # [ 263.438160] gitaly[911]: time="2026-11-10T09:54:13.661Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AZ6C1Z5DS898403TPP9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.649" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2077gitlab # [ 263.438905] gitlab-workhorse[825]: time="2026-11-10T09:54:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162078gitlab # [ 263.441541] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 562079gitlab # [ 263.442205] gitaly[911]: time="2026-11-10T09:54:13.662Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AZ6C1Z5DS898403TPP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:13.649" grpc.time_ms=12.862 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2080gitlab # [ 263.444356] gitaly[911]: time="2026-11-10T09:54:13.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AZVS51M4FV008M4BAMJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:43.568" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:13.668" grpc.time_ms=0.638 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2081gitlab # [ 263.461501] gitaly[911]: time="2026-11-10T09:54:13.686Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:13.686" grpc.time_ms=0.272 pid=911 span.kind=server system=grpc2082gitlab # [ 263.519216] gitaly[911]: time="2026-11-10T09:54:13.744Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:13.743" grpc.time_ms=0.478 pid=911 span.kind=server system=grpc2083gitlab # [ 263.531835] gitaly[911]: time="2026-11-10T09:54:13.756Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E2AZVS51M4FV008M4BAMJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:23.259" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.749" grpc.time_ms=7.493 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2084gitlab # [ 263.564564] gitaly[911]: time="2026-11-10T09:54:13.788Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:13.788" grpc.time_ms=0.341 pid=911 span.kind=server system=grpc2085gitlab # [ 263.574694] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1692086gitlab # [ 263.575570] gitaly[911]: time="2026-11-10T09:54:13.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:43.597" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:13.797" grpc.time_ms=1.063 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087gitlab # [ 263.584207] gitaly[911]: time="2026-11-10T09:54:13.809Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 diskcache=8fac56df-b7b7-4d5f-ac67-09026e1337fe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T09:54:43.666" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.666" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088gitlab # [ 263.587180] gitaly[911]: time="2026-11-10T09:54:13.809Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=89 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7338 command.oublock=0 command.real_time_ms=134 command.spawn_token_fork_ms=0 command.system_time_ms=61 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T09:54:43.666" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.666" grpc.time_ms=142.379 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2089gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a'2090gitlab # [ 263.588273] gitaly[911]: time="2026-11-10T09:54:13.813Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T09:54:43.605" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.805" grpc.time_ms=7.547 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091gitlab # [ 263.624527] gitaly[911]: time="2026-11-10T09:54:13.849Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B5E535CQWCX24Y6R0YX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:43.048" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:13.848" grpc.time_ms=0.437 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092gitlab # [ 263.627559] gitaly[911]: time="2026-11-10T09:54:13.852Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T09:54:43.851" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:13.851" grpc.time_ms=0.464 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', in 0.05 seconds)2094(finished: subtest: push delivers the webhook, in 51.89 seconds)2095subtest: eval and build post commit statuses2096gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2097gitlab # [ 263.652938] gitaly[911]: time="2026-11-10T09:54:13.877Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:43.875" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:13.876" grpc.time_ms=1.625 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab # [ 263.659061] nixbot[2700]: {"ts": "2026-11-10T09:54:13.883Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:51046 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1\" 200"}2099gitlab # [ 263.665116] gitaly[911]: time="2026-11-10T09:54:13.889Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2B5E535CQWCX24Y6R0YX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:23.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.882" grpc.time_ms=7.487 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab # [ 263.667271] gitaly[911]: time="2026-11-10T09:54:13.892Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:23.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.881" grpc.time_ms=10.829 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab # [ 263.686658] gitaly[911]: time="2026-11-10T09:54:13.911Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T09:54:23.030" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:13.911" grpc.time_ms=0.189 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102gitlab # [ 263.703138] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.926Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"affa4236db8417c3a4e4d799","created_at":"2026-11-10T09:54:13.923Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"8eadb050b69cb845ec61f9cd","job_size_bytes":3,"pid":2576,"job_status":"deduplicated","message":"PipelineProcessWorker JID-affa4236db8417c3a4e4d799: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2103gitlab # [ 263.704943] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8eadb050b69cb845ec61f9cd","created_at":"2026-11-10T09:54:13.602Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-26c000337a5f45a7","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.606Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2576,"job_status":"rescheduled","message":"PipelineProcessWorker JID-8eadb050b69cb845ec61f9cd: rescheduled"}2104gitlab # [ 263.710804] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.935Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8eadb050b69cb845ec61f9cd","created_at":"2026-11-10T09:54:13.602Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-26c000337a5f45a7","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.606Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8eadb050b69cb845ec61f9cd: done: 0.327432 sec","job_status":"done","queue_duration_s":0.001855,"scheduling_latency_s":0.001891,"gitaly_calls":3,"gitaly_duration_s":0.155099,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.015576,"redis_read_bytes":226,"redis_write_bytes":18468,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000819,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000701,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002966,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.008235,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000255,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000418,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116662,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.256455499,"duration_s":0.327432,"completed_at":"2026-11-10T09:54:13.935Z","load_balancing_strategy":"primary","db_duration_s":0.033934,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2105gitlab # [ 263.758299] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.936Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2d013f1f0b1e165187e9e546","created_at":"2026-11-10T09:54:13.657Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.659Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2d013f1f0b1e165187e9e546: start","job_status":"start","queue_duration_s":0.276656,"scheduling_latency_s":0.276685}2106gitlab # [ 263.758914] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2d013f1f0b1e165187e9e546","created_at":"2026-11-10T09:54:13.657Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.659Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2d013f1f0b1e165187e9e546: done: 0.043956 sec","job_status":"done","queue_duration_s":0.276656,"scheduling_latency_s":0.276685,"redis_calls":7,"redis_duration_s":0.001994,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043956,"completed_at":"2026-11-10T09:54:13.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014717,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2107gitlab # [ 263.826668] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/64ce515318877fcd7b1ffb60d5f22da640a32041 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1362108gitlab # [ 263.827464] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 472109gitlab # [ 263.827952] gitaly[911]: time="2026-11-10T09:54:14.024Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2BAS1QWQ0AH8JQEC7M96 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.017" grpc.time_ms=7.578 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2110gitlab # [ 263.828661] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:13.984Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a75e5be53a73b9ddbab0178b","created_at":"2026-11-10T09:54:13.815Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.817Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a75e5be53a73b9ddbab0178b: start","job_status":"start","queue_duration_s":0.166253,"scheduling_latency_s":0.166281}2111gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 0.22 seconds)2112gitlab # [ 263.829478] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.010Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a75e5be53a73b9ddbab0178b","created_at":"2026-11-10T09:54:13.815Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.817Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a75e5be53a73b9ddbab0178b: done: 0.026496 sec","job_status":"done","queue_duration_s":0.166253,"scheduling_latency_s":0.166281,"redis_calls":7,"redis_duration_s":0.001103,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026496,"completed_at":"2026-11-10T09:54:14.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113gitlab # [ 263.873397] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.011Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"cb81fd07beab53641da2f652","created_at":"2026-11-10T09:54:13.823Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.824Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-cb81fd07beab53641da2f652: start","job_status":"start","queue_duration_s":0.187325,"scheduling_latency_s":0.187343}2114gitlab # [ 263.914270] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"cb81fd07beab53641da2f652","created_at":"2026-11-10T09:54:13.823Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.824Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-cb81fd07beab53641da2f652: done: 0.035157 sec","job_status":"done","queue_duration_s":0.187325,"scheduling_latency_s":0.187343,"redis_calls":7,"redis_duration_s":0.013998,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001179,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035157,"completed_at":"2026-11-10T09:54:14.047Z","load_balancing_strategy":"primary","db_duration_s":0.00343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115gitlab # [ 263.915838] gitaly[911]: time="2026-11-10T09:54:14.057Z" level=info msg="finished unary call with code OK" catfile.duration_ms=37 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=37 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B9ADBSYPH2ZJKVA9GBT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:44.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:14.019" grpc.time_ms=37.425 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 263.926147] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 1702117gitlab # [ 263.928647] gitlab-workhorse[825]: time="2026-11-10T09:54:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672118gitlab # [ 263.932081] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1162119gitlab # [ 263.932318] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.048Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"7232ecf518fb1984525ad073","created_at":"2026-11-10T09:54:13.829Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.830Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7232ecf518fb1984525ad073: start","job_status":"start","queue_duration_s":0.217136,"scheduling_latency_s":0.217154}2120gitlab # [ 263.977232] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.068Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"7232ecf518fb1984525ad073","created_at":"2026-11-10T09:54:13.829Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.830Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7232ecf518fb1984525ad073: done: 0.020307 sec","job_status":"done","queue_duration_s":0.217136,"scheduling_latency_s":0.217154,"redis_calls":10,"redis_duration_s":0.002134,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001136,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020307,"completed_at":"2026-11-10T09:54:14.068Z","load_balancing_strategy":"primary","db_duration_s":0.003058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 263.987293] gitaly[911]: time="2026-11-10T09:54:14.143Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2BCA45M12QNAJNH7R7RS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.134" pid=911 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2122gitlab # [ 264.033566] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.069Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"45a89fcfea5f2da700462aec","created_at":"2026-11-10T09:54:13.831Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.832Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-45a89fcfea5f2da700462aec: start","job_status":"start","queue_duration_s":0.237263,"scheduling_latency_s":0.23728}2123gitlab # [ 264.034448] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.086Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"45a89fcfea5f2da700462aec","created_at":"2026-11-10T09:54:13.831Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.832Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-45a89fcfea5f2da700462aec: done: 0.017028 sec","job_status":"done","queue_duration_s":0.237263,"scheduling_latency_s":0.23728,"redis_calls":7,"redis_duration_s":0.0011099999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017028,"completed_at":"2026-11-10T09:54:14.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 264.080406] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 542125gitlab # [ 264.081062] gitlab-workhorse[825]: time="2026-11-10T09:54:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672126gitlab # [ 264.081560] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 542127gitlab # [ 264.082126] gitaly[911]: time="2026-11-10T09:54:14.143Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2BCA45M12QNAJNH7R7RS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.134" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2128gitlab # [ 264.082785] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.088Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c75a08dfb0611cce3de008ec","created_at":"2026-11-10T09:54:13.833Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.835Z","job_size_bytes":19,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c75a08dfb0611cce3de008ec: start","job_status":"start","queue_duration_s":0.252408,"scheduling_latency_s":0.252425}2129gitlab # [ 264.083949] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c75a08dfb0611cce3de008ec","created_at":"2026-11-10T09:54:13.833Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.835Z","job_size_bytes":19,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c75a08dfb0611cce3de008ec: done: 0.018855 sec","job_status":"done","queue_duration_s":0.252408,"scheduling_latency_s":0.252425,"redis_calls":8,"redis_duration_s":0.001594,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018855,"completed_at":"2026-11-10T09:54:14.107Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130gitlab # [ 264.127157] gitaly[911]: time="2026-11-10T09:54:14.143Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2BCA45M12QNAJNH7R7RS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.134" grpc.time_ms=8.599 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2131gitlab # [ 264.137593] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.108Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2d639ec50410162d1bfc7363","created_at":"2026-11-10T09:54:13.836Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.837Z","job_size_bytes":66,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2d639ec50410162d1bfc7363: start","job_status":"start","queue_duration_s":0.270543,"scheduling_latency_s":0.270567}2132gitlab # [ 264.153885] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.145Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2d639ec50410162d1bfc7363","created_at":"2026-11-10T09:54:13.836Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.837Z","job_size_bytes":66,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2d639ec50410162d1bfc7363: done: 0.036514 sec","job_status":"done","queue_duration_s":0.270543,"scheduling_latency_s":0.270567,"redis_calls":7,"redis_duration_s":0.001147,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036514,"completed_at":"2026-11-10T09:54:14.144Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 264.194942] gitaly[911]: time="2026-11-10T09:54:14.224Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72E2BGTC02P4FA8Q8XP5JVP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.211" grpc.time_ms=12.907 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2134gitlab # [ 264.236116] gitaly[911]: time="2026-11-10T09:54:14.281Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2BJMT5KQRE5AEQWMP8X5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.274" pid=911 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc2135gitlab # [ 264.236882] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.146Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1f065a7d9167bfbba90e567f","created_at":"2026-11-10T09:54:13.838Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.839Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1f065a7d9167bfbba90e567f: start","job_status":"start","queue_duration_s":0.306666,"scheduling_latency_s":0.306714}2136gitlab # [ 264.237516] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.180Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1f065a7d9167bfbba90e567f","created_at":"2026-11-10T09:54:13.838Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.839Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1f065a7d9167bfbba90e567f: done: 0.034289 sec","job_status":"done","queue_duration_s":0.306666,"scheduling_latency_s":0.306714,"redis_calls":7,"redis_duration_s":0.009079,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00412,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034289,"completed_at":"2026-11-10T09:54:14.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2137gitlab # [ 264.285368] gitaly[911]: time="2026-11-10T09:54:14.281Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2BJMT5KQRE5AEQWMP8X5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.274" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2138gitlab # [ 264.286142] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c1f7d2610e9f1aee73254bd3","created_at":"2026-11-10T09:54:13.839Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.840Z","job_size_bytes":6,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c1f7d2610e9f1aee73254bd3: start","job_status":"start","queue_duration_s":0.342183,"scheduling_latency_s":0.342208}2139gitlab # [ 264.297098] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c1f7d2610e9f1aee73254bd3","created_at":"2026-11-10T09:54:13.839Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.840Z","job_size_bytes":6,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c1f7d2610e9f1aee73254bd3: done: 0.010655 sec","job_status":"done","queue_duration_s":0.342183,"scheduling_latency_s":0.342208,"redis_calls":7,"redis_duration_s":0.000932,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010655,"completed_at":"2026-11-10T09:54:14.193Z","load_balancing_strategy":"primary","db_duration_s":0.000353,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 264.363065] gitaly[911]: time="2026-11-10T09:54:14.283Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2BJMT5KQRE5AEQWMP8X5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:14.274" grpc.time_ms=7.999 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2141gitlab # [ 264.363796] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.194Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1397527bd6fd23bd0d313862","created_at":"2026-11-10T09:54:13.841Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.842Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1397527bd6fd23bd0d313862: start","job_status":"start","queue_duration_s":0.352042,"scheduling_latency_s":0.352057}2142gitlab # [ 264.364573] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.211Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1397527bd6fd23bd0d313862","created_at":"2026-11-10T09:54:13.841Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.842Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1397527bd6fd23bd0d313862: done: 0.017245 sec","job_status":"done","queue_duration_s":0.352042,"scheduling_latency_s":0.352057,"redis_calls":7,"redis_duration_s":0.0024460000000000003,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017245,"completed_at":"2026-11-10T09:54:14.211Z","load_balancing_strategy":"primary","db_duration_s":0.002981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 264.408589] gitaly[911]: time="2026-11-10T09:54:14.299Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:14.299" grpc.time_ms=0.298 pid=911 span.kind=server system=grpc2144gitlab # [ 264.410997] gitaly[911]: time="2026-11-10T09:54:14.320Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:14.320" grpc.time_ms=0.239 pid=911 span.kind=server system=grpc2145gitlab # [ 264.455990] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.212Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"2a9e46fa6ddca370c42dbd0b","created_at":"2026-11-10T09:54:13.929Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.930Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2a9e46fa6ddca370c42dbd0b: start","job_status":"start","queue_duration_s":0.282163,"scheduling_latency_s":0.282178}2146gitlab # [ 264.456686] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.411Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"2a9e46fa6ddca370c42dbd0b","created_at":"2026-11-10T09:54:13.929Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5c6f39d19e8249e6","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:13.930Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2a9e46fa6ddca370c42dbd0b: done: 0.198624 sec","job_status":"done","queue_duration_s":0.282163,"scheduling_latency_s":0.282178,"gitaly_calls":3,"gitaly_duration_s":0.083279,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.010657,"redis_read_bytes":141,"redis_write_bytes":16719,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000588,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000606,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001929,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11225,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.004047,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000158,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00035,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002391,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081188,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.170961113,"duration_s":0.198624,"completed_at":"2026-11-10T09:54:14.411Z","load_balancing_strategy":"primary","db_duration_s":0.024316,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2147gitlab # [ 264.499588] gitaly[911]: time="2026-11-10T09:54:14.340Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:14.339" grpc.time_ms=0.377 pid=911 span.kind=server system=grpc2148gitlab # [ 264.499893] gitaly[911]: time="2026-11-10T09:54:14.349Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 diskcache=b7138b84-419e-4bca-bc5a-aa80896a7eb5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T09:54:44.272" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:14.273" pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149gitlab # [ 264.500660] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.412Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7af1b1e49e745caa4454ad15","created_at":"2026-11-10T09:54:14.264Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.267Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7af1b1e49e745caa4454ad15: start","job_status":"start","queue_duration_s":0.145048,"scheduling_latency_s":0.145064}2150gitlab # [ 264.514906] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.436Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7af1b1e49e745caa4454ad15","created_at":"2026-11-10T09:54:14.264Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.267Z","job_size_bytes":23,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7af1b1e49e745caa4454ad15: done: 0.024299 sec","job_status":"done","queue_duration_s":0.145048,"scheduling_latency_s":0.145064,"redis_calls":7,"redis_duration_s":0.000958,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024299,"completed_at":"2026-11-10T09:54:14.436Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004232,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2151gitlab # [ 264.563190] gitaly[911]: time="2026-11-10T09:54:14.349Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7209 command.oublock=0 command.real_time_ms=69 command.spawn_token_fork_ms=2 command.system_time_ms=46 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T09:54:44.272" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:14.273" grpc.time_ms=76.51 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2152gitlab # [ 264.566588] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.437Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e705510f0ecc81ddb6ec1174","created_at":"2026-11-10T09:54:14.354Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.356Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e705510f0ecc81ddb6ec1174: start","job_status":"start","queue_duration_s":0.081123,"scheduling_latency_s":0.081138}2153gitlab # [ 264.613064] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.453Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e705510f0ecc81ddb6ec1174","created_at":"2026-11-10T09:54:14.354Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.356Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e705510f0ecc81ddb6ec1174: done: 0.015562 sec","job_status":"done","queue_duration_s":0.081123,"scheduling_latency_s":0.081138,"redis_calls":7,"redis_duration_s":0.0009159999999999999,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015562,"completed_at":"2026-11-10T09:54:14.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 264.659295] gitaly[911]: time="2026-11-10T09:54:14.385Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T09:54:44.383" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:14.384" grpc.time_ms=0.737 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2155gitlab # [ 264.659997] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.454Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"48464de7a37f333504d86293","created_at":"2026-11-10T09:54:14.361Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.363Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-48464de7a37f333504d86293: start","job_status":"start","queue_duration_s":0.090924,"scheduling_latency_s":0.090939}2156gitlab # [ 264.660691] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.470Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"48464de7a37f333504d86293","created_at":"2026-11-10T09:54:14.361Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.363Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-48464de7a37f333504d86293: done: 0.016427 sec","job_status":"done","queue_duration_s":0.090924,"scheduling_latency_s":0.090939,"redis_calls":7,"redis_duration_s":0.0009170000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016427,"completed_at":"2026-11-10T09:54:14.470Z","load_balancing_strategy":"primary","db_duration_s":0.002622,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2157gitlab # [ 264.707657] gitaly[911]: time="2026-11-10T09:54:14.403Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2AR7VTCVP3TZT12TYK14 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:44.402" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:14.403" grpc.time_ms=0.448 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158gitlab # [ 264.708496] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.471Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"511cecd5000c448e87c5474b","created_at":"2026-11-10T09:54:14.368Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.369Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-511cecd5000c448e87c5474b: start","job_status":"start","queue_duration_s":0.102238,"scheduling_latency_s":0.102253}2159gitlab # [ 264.749929] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.490Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"511cecd5000c448e87c5474b","created_at":"2026-11-10T09:54:14.368Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.369Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-511cecd5000c448e87c5474b: done: 0.019456 sec","job_status":"done","queue_duration_s":0.102238,"scheduling_latency_s":0.102253,"redis_calls":10,"redis_duration_s":0.001289,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000394,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019456,"completed_at":"2026-11-10T09:54:14.490Z","load_balancing_strategy":"primary","db_duration_s":0.003994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 264.753060] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.491Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"82635528d3a19c553567d28a","created_at":"2026-11-10T09:54:14.369Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.370Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-82635528d3a19c553567d28a: start","job_status":"start","queue_duration_s":0.121363,"scheduling_latency_s":0.121379}2161gitlab # [ 264.769207] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"82635528d3a19c553567d28a","created_at":"2026-11-10T09:54:14.369Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.370Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-82635528d3a19c553567d28a: done: 0.074834 sec","job_status":"done","queue_duration_s":0.121363,"scheduling_latency_s":0.121379,"redis_calls":7,"redis_duration_s":0.000913,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074834,"completed_at":"2026-11-10T09:54:14.566Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162gitlab # [ 264.818942] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7c54db837d687ecd98201ad3","created_at":"2026-11-10T09:54:14.370Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.372Z","job_size_bytes":19,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7c54db837d687ecd98201ad3: start","job_status":"start","queue_duration_s":0.195782,"scheduling_latency_s":0.195802}2163gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2164gitlab # [ 264.824423] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.585Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7c54db837d687ecd98201ad3","created_at":"2026-11-10T09:54:14.370Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.372Z","job_size_bytes":19,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7c54db837d687ecd98201ad3: done: 0.017346 sec","job_status":"done","queue_duration_s":0.195782,"scheduling_latency_s":0.195802,"redis_calls":8,"redis_duration_s":0.001113,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017346,"completed_at":"2026-11-10T09:54:14.585Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2165gitlab # [ 264.879911] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.586Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2d9daaef0fb06de3c2391c1c","created_at":"2026-11-10T09:54:14.372Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.373Z","job_size_bytes":66,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2d9daaef0fb06de3c2391c1c: start","job_status":"start","queue_duration_s":0.21242,"scheduling_latency_s":0.212435}2166gitlab # [ 264.880723] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.599Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2d9daaef0fb06de3c2391c1c","created_at":"2026-11-10T09:54:14.372Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.373Z","job_size_bytes":66,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2d9daaef0fb06de3c2391c1c: done: 0.013712 sec","job_status":"done","queue_duration_s":0.21242,"scheduling_latency_s":0.212435,"redis_calls":7,"redis_duration_s":0.00092,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013712,"completed_at":"2026-11-10T09:54:14.599Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2167gitlab # [ 264.943783] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.600Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"333df7135c68041596257f47","created_at":"2026-11-10T09:54:14.374Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.375Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-333df7135c68041596257f47: start","job_status":"start","queue_duration_s":0.225842,"scheduling_latency_s":0.225857}2168gitlab # [ 264.944880] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.620Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"333df7135c68041596257f47","created_at":"2026-11-10T09:54:14.374Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.375Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-333df7135c68041596257f47: done: 0.019143 sec","job_status":"done","queue_duration_s":0.225842,"scheduling_latency_s":0.225857,"redis_calls":7,"redis_duration_s":0.00105,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019143,"completed_at":"2026-11-10T09:54:14.620Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 264.991096] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.620Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6b178d950b532e2d7bfefebd","created_at":"2026-11-10T09:54:14.375Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.376Z","job_size_bytes":6,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6b178d950b532e2d7bfefebd: start","job_status":"start","queue_duration_s":0.244747,"scheduling_latency_s":0.244762}2170gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 0.19 seconds)2171gitlab # [ 264.995748] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.630Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6b178d950b532e2d7bfefebd","created_at":"2026-11-10T09:54:14.375Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.376Z","job_size_bytes":6,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6b178d950b532e2d7bfefebd: done: 0.009382 sec","job_status":"done","queue_duration_s":0.244747,"scheduling_latency_s":0.244762,"redis_calls":7,"redis_duration_s":0.000904,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009382,"completed_at":"2026-11-10T09:54:14.630Z","load_balancing_strategy":"primary","db_duration_s":0.000434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 265.061935] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.631Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"807ae70959c514a7c9ee1425","created_at":"2026-11-10T09:54:14.376Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.377Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-807ae70959c514a7c9ee1425: start","job_status":"start","queue_duration_s":0.253891,"scheduling_latency_s":0.253905}2173gitlab # [ 265.063194] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:14.646Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"807ae70959c514a7c9ee1425","created_at":"2026-11-10T09:54:14.376Z","trace_propagation_headers":{"sentry-trace":"b4e3ef1bdb6144a79b5328dc7d4e0fc4-5b57e0fd4af147a9","baggage":"sentry-trace_id=b4e3ef1bdb6144a79b5328dc7d4e0fc4,sentry-release=Unknown"},"correlation_id":"01M72E2AR7VTCVP3TZT12TYK14","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:14.377Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-807ae70959c514a7c9ee1425: done: 0.015229 sec","job_status":"done","queue_duration_s":0.253891,"scheduling_latency_s":0.253905,"redis_calls":7,"redis_duration_s":0.0009580000000000001,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015229,"completed_at":"2026-11-10T09:54:14.646Z","load_balancing_strategy":"primary","db_duration_s":0.002176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174gitlab # [ 265.127982] gitaly[911]: time="2026-11-10T09:54:15.276Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2CFEE19QV8C9EHGV2CPY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:45.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:15.275" grpc.time_ms=0.848 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175gitlab # [ 265.129633] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 1402176gitlab # [ 265.160769] gitaly[911]: time="2026-11-10T09:54:15.385Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=114688 command.minflt=623 command.oublock=0 command.real_time_ms=1470 command.spawn_token_fork_ms=12 command.system_time_ms=43 command.user_time_ms=15 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T09:54:23.033" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:13.913" grpc.time_ms=1471.549 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2177gitlab # [ 265.206215] gitaly[911]: time="2026-11-10T09:54:15.431Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:15.430" grpc.time_ms=0.584 pid=911 span.kind=server system=grpc2178gitlab # [ 265.239705] gitaly[911]: time="2026-11-10T09:54:15.464Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:15.464" grpc.time_ms=0.393 pid=911 span.kind=server system=grpc2179gitlab # [ 265.252704] gitaly[911]: time="2026-11-10T09:54:15.477Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:25.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:15.398" operations="[repack_refs]" pid=911 ref=refs/keep-around/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a remote_ip="2001:db8:1::1" revision=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a span.kind=server system=grpc username=root2180gitlab # [ 265.254468] gitaly[911]: time="2026-11-10T09:54:15.479Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:25.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:15.398" pid=911 ref=refs/keep-around/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-10 09:53:10.806444232 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a span.kind=server system=grpc username=root2181gitlab # [ 265.259159] gitaly[911]: time="2026-11-10T09:54:15.484Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 diskcache=0eb9dc22-7035-4123-a823-7837b03c3c63 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:25.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:15.398" pid=911 ref=refs/keep-around/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a remote_ip="2001:db8:1::1" revision=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a span.kind=server system=grpc username=root2182gitlab # [ 265.259971] gitaly[911]: time="2026-11-10T09:54:15.484Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=51 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5565 command.oublock=8 command.real_time_ms=77 command.spawn_token_fork_ms=0 command.system_time_ms=32 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:25.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:15.398" grpc.time_ms=85.684 pid=911 ref=refs/keep-around/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a remote_ip="2001:db8:1::1" revision=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a span.kind=server system=grpc username=root2183gitlab # [ 265.262846] gitaly[911]: time="2026-11-10T09:54:15.487Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:45.287" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:15.487" grpc.time_ms=0.333 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2184gitlab # [ 265.272879] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.494Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ee1622cd509120f3cde54e7a","created_at":"2026-11-10T09:54:15.489Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-a0b2420743cf446e","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.493Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ee1622cd509120f3cde54e7a: start","job_status":"start","queue_duration_s":0.001364,"scheduling_latency_s":0.0014}2185gitlab # [ 265.322077] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.529Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ee1622cd509120f3cde54e7a","created_at":"2026-11-10T09:54:15.489Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-a0b2420743cf446e","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.493Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ee1622cd509120f3cde54e7a: done: 0.034636 sec","job_status":"done","queue_duration_s":0.001364,"scheduling_latency_s":0.0014,"redis_calls":7,"redis_duration_s":0.002538,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034636,"completed_at":"2026-11-10T09:54:15.529Z","load_balancing_strategy":"primary","db_duration_s":0.005095,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2186gitlab # [ 265.329208] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.531Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1f69e173b45bcef2f3fb7564","created_at":"2026-11-10T09:54:15.494Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-a0b2420743cf446e","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.496Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1f69e173b45bcef2f3fb7564: start","job_status":"start","queue_duration_s":0.034737,"scheduling_latency_s":0.034758}2187gitlab # [ 265.381181] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.582Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1f69e173b45bcef2f3fb7564","created_at":"2026-11-10T09:54:15.494Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-a0b2420743cf446e","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.496Z","job_size_bytes":65,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1f69e173b45bcef2f3fb7564: done: 0.0516 sec","job_status":"done","queue_duration_s":0.034737,"scheduling_latency_s":0.034758,"redis_calls":10,"redis_duration_s":0.002022,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0516,"completed_at":"2026-11-10T09:54:15.582Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188gitlab # [ 265.386474] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.584Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c89c9d089b9cc443abc8d276","created_at":"2026-11-10T09:54:15.568Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-a0b2420743cf446e","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.572Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c89c9d089b9cc443abc8d276: start","job_status":"start","queue_duration_s":0.011149,"scheduling_latency_s":0.011169}2189gitlab # [ 265.436066] nixbot[2700]: {"ts": "2026-11-10T09:54:15.596Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-a59288aee385-4"}2190gitlab # [ 265.436753] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 18312191gitlab # [ 265.437835] gitaly[911]: time="2026-11-10T09:54:15.635Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:25.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:15.398" pid=911 ref=refs/keep-around/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a remote_ip="2001:db8:1::1" revision=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2192gitlab # [ 265.438606] gitaly[911]: time="2026-11-10T09:54:15.635Z" level=info msg="optimized repository" clean_stale_data_time=155.539042ms clean_work_trees_time="81.854µs" commit_graph_time="3.912µs" correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.632µs" pid=911 prune_time="3.073µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.47µs" storage_name=default system=housekeeping total_time=155.738788ms2193gitlab # [ 265.439188] gitaly[911]: time="2026-11-10T09:54:15.635Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T09:54:25.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T09:54:15.398" pid=911 ref=refs/keep-around/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a remote_ip="2001:db8:1::1" revision=a59288aee385d50e3b1a10cc2d2c3866b2c77c0a span.kind=server system=grpc username=root2194gitlab # [ 265.439845] gitaly[911]: time="2026-11-10T09:54:15.644Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T09:54:45.643" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:15.643" grpc.time_ms=0.934 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2195gitlab # [ 265.454772] gitaly[911]: time="2026-11-10T09:54:15.679Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2B3SEM6KEJ624T3FA6A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:45.678" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:15.678" grpc.time_ms=0.665 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2196gitlab # [ 265.466138] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.690Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c89c9d089b9cc443abc8d276","created_at":"2026-11-10T09:54:15.568Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-a0b2420743cf446e","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.572Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c89c9d089b9cc443abc8d276: done: 0.106115 sec","job_status":"done","queue_duration_s":0.011149,"scheduling_latency_s":0.011169,"gitaly_calls":2,"gitaly_duration_s":0.006537,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.005291,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000701,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000971,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000257,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000159,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000436,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074145,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.083075921,"duration_s":0.106115,"completed_at":"2026-11-10T09:54:15.690Z","load_balancing_strategy":"primary","db_duration_s":0.025148,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2197gitlab # [ 265.511375] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.692Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c888426d58501b730f999838","created_at":"2026-11-10T09:54:15.625Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-36a504551cad41c0","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.626Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c888426d58501b730f999838: start","job_status":"start","queue_duration_s":0.065056,"scheduling_latency_s":0.065076}2198gitlab # [ 265.554683] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:15.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c888426d58501b730f999838","created_at":"2026-11-10T09:54:15.625Z","trace_propagation_headers":{"sentry-trace":"f87993900bb24ed7a6fb97e01cb042df-36a504551cad41c0","baggage":"sentry-trace_id=f87993900bb24ed7a6fb97e01cb042df,sentry-release=Unknown"},"correlation_id":"01M72E2B3SEM6KEJ624T3FA6A0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:15.626Z","job_size_bytes":3,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c888426d58501b730f999838: done: 0.02628 sec","job_status":"done","queue_duration_s":0.065056,"scheduling_latency_s":0.065076,"redis_calls":7,"redis_duration_s":0.00117,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02628,"completed_at":"2026-11-10T09:54:15.718Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199gitlab # [ 265.604102] nixbot[2700]: {"ts": "2026-11-10T09:54:15.693Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a59288aee385-4", "argv0": "bwrap"}2200gitlab # [ 265.676882] nix-daemon[2865]: accepted connection from pid 3778, user nixbot2201gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2202gitlab # [ 266.146858] gitaly[911]: time="2026-11-10T09:54:16.371Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2DMF4ZWDFHCXQKW0TQXM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:46.269" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:16.370" grpc.time_ms=0.836 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2203gitlab # [ 266.168184] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 552204gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 0.10 seconds)2205gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2206gitlab # [ 267.258357] gitaly[911]: time="2026-11-10T09:54:17.482Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2EPAHKR1NTB85S3QM8PN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:47.280" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:17.481" grpc.time_ms=0.949 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2207gitlab # [ 267.303427] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1082208gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 0.14 seconds)2209gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2210gitlab # [ 268.426357] gitaly[911]: time="2026-11-10T09:54:18.650Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2FT8H5CAVPDY8YMFE4YK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:48.549" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:18.650" grpc.time_ms=0.554 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2211gitlab # [ 268.473403] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1282212gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 0.17 seconds)2213gitlab # [ 268.899579] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"73f91181123d1099086983ad","created_at":"2026-11-10T09:54:19.103Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb8ffc8dde97554f316b289588f1525c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.120Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-73f91181123d1099086983ad: start","job_status":"start","queue_duration_s":0.003189,"scheduling_latency_s":0.003217}2214gitlab # [ 268.984438] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"73f91181123d1099086983ad","created_at":"2026-11-10T09:54:19.103Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb8ffc8dde97554f316b289588f1525c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.120Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-73f91181123d1099086983ad: done: 0.084822 sec","job_status":"done","queue_duration_s":0.003189,"scheduling_latency_s":0.003217,"redis_calls":11,"redis_duration_s":0.008992,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003065,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002679,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084822,"completed_at":"2026-11-10T09:54:19.208Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2215gitlab # [ 268.987932] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"62988bf13c73843000688171","created_at":"2026-11-10T09:54:19.153Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"452fd04899309b432dbbaa2dc715e4f9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.167Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-62988bf13c73843000688171: start","job_status":"start","queue_duration_s":0.04495,"scheduling_latency_s":0.045003}2216gitlab # [ 269.055111] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"62988bf13c73843000688171","created_at":"2026-11-10T09:54:19.153Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"452fd04899309b432dbbaa2dc715e4f9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.167Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-62988bf13c73843000688171: done: 0.06765 sec","job_status":"done","queue_duration_s":0.04495,"scheduling_latency_s":0.045003,"redis_calls":13,"redis_duration_s":0.004667,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00127,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000779708,"exclusive_lock_hold_duration_s":0.034599369,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.06765,"completed_at":"2026-11-10T09:54:19.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217gitlab # [ 269.126955] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f013b31e9027d039a5f426cc","created_at":"2026-11-10T09:54:19.179Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc1decda52b61323679b7d9b62e627f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.348Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f013b31e9027d039a5f426cc: start","job_status":"start","queue_duration_s":0.002594,"scheduling_latency_s":0.002644}2218gitlab # [ 269.210834] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.435Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2219gitlab # [ 269.253632] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f013b31e9027d039a5f426cc","created_at":"2026-11-10T09:54:19.179Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc1decda52b61323679b7d9b62e627f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.348Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f013b31e9027d039a5f426cc: done: 0.126596 sec","job_status":"done","queue_duration_s":0.002594,"scheduling_latency_s":0.002644,"redis_calls":9,"redis_duration_s":0.026732,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001368,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.023204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00216,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.126596,"completed_at":"2026-11-10T09:54:19.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000789,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 269.330624] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2d984b57ad31e48bd32b9f89","created_at":"2026-11-10T09:54:19.409Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16df0f7e5064e4267919b756adf363b5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.427Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2d984b57ad31e48bd32b9f89: start","job_status":"start","queue_duration_s":0.127823,"scheduling_latency_s":0.127873}2221gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2222gitlab # [ 269.459323] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2d984b57ad31e48bd32b9f89","created_at":"2026-11-10T09:54:19.409Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16df0f7e5064e4267919b756adf363b5","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.427Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2d984b57ad31e48bd32b9f89: done: 0.129057 sec","job_status":"done","queue_duration_s":0.127823,"scheduling_latency_s":0.127873,"redis_calls":9,"redis_duration_s":0.011702,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001996,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003183,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.129057,"completed_at":"2026-11-10T09:54:19.683Z","load_balancing_strategy":"primary","db_duration_s":0.048197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223gitlab # [ 269.502464] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:19.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ef98f31bf06d68e6b01c2345","created_at":"2026-11-10T09:54:19.461Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.480Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ef98f31bf06d68e6b01c2345: start","job_status":"start","queue_duration_s":0.245932,"scheduling_latency_s":0.245958}2224gitlab # [ 269.596835] gitaly[911]: time="2026-11-10T09:54:19.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2GYZCEQX9D710NE2ZH7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:49.619" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:19.820" grpc.time_ms=0.724 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2225gitlab # [ 269.625364] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1062226gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 0.15 seconds)2227gitlab # [ 269.755809] nix-daemon[2865]: accepted connection from pid 3815, user nixbot2228gitlab # [ 269.766749] nix-daemon[2865]: accepted connection from pid 3778, user nixbot2229gitlab # [ 269.814810] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ef98f31bf06d68e6b01c2345","created_at":"2026-11-10T09:54:19.461Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.480Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ef98f31bf06d68e6b01c2345: done: 0.31311 sec","job_status":"done","queue_duration_s":0.245932,"scheduling_latency_s":0.245958,"redis_calls":70,"redis_duration_s":0.102283,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.015562,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001278,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.080548,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004895,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.31311,"completed_at":"2026-11-10T09:54:20.039Z","load_balancing_strategy":"primary","db_duration_s":0.009515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2230gitlab # [ 269.821098] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"394a974cbaec5f0f1ee7f500","created_at":"2026-11-10T09:54:19.582Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2c071b480e61a8580298a61a3ccaf66","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.600Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-394a974cbaec5f0f1ee7f500: start","job_status":"start","queue_duration_s":0.445429,"scheduling_latency_s":0.445458}2231gitlab # [ 269.844765] gitaly[911]: time="2026-11-10T09:54:20.069Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2H7MR6JEE0397YZGKNXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:50.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:20.068" grpc.time_ms=0.681 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2232gitlab # [ 269.880025] gitaly[911]: time="2026-11-10T09:54:20.104Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E2H7MR6JEE0397YZGKNXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:30.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:20.098" grpc.time_ms=6.292 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2233gitlab # [ 269.884190] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"394a974cbaec5f0f1ee7f500","created_at":"2026-11-10T09:54:19.582Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2c071b480e61a8580298a61a3ccaf66","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.600Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-394a974cbaec5f0f1ee7f500: done: 0.062792 sec","job_status":"done","queue_duration_s":0.445429,"scheduling_latency_s":0.445458,"redis_calls":12,"redis_duration_s":0.011656000000000001,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009919,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015937221,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.062792,"completed_at":"2026-11-10T09:54:20.108Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010959,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2234gitlab # [ 269.945868] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bf3d502c2e1e490241e098b6","created_at":"2026-11-10T09:54:19.643Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a9eedf03e9fb3b8e56ef1fc942189a2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.658Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bf3d502c2e1e490241e098b6: start","job_status":"start","queue_duration_s":0.463879,"scheduling_latency_s":0.46392}2235gitlab # [ 269.946937] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1352236gitlab # [ 269.987247] gitaly[911]: time="2026-11-10T09:54:20.211Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2HC4WDZ1ZZ5DR8JWZR4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:50.010" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:20.211" grpc.time_ms=0.609 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237gitlab # [ 270.025076] gitaly[911]: time="2026-11-10T09:54:20.249Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72E2HC4WDZ1ZZ5DR8JWZR4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:30.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:20.243" grpc.time_ms=6.125 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2238gitlab # [ 270.069280] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1292239gitlab # [ 270.123128] gitaly[911]: time="2026-11-10T09:54:20.348Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2HGK6J1DW2JC40C3TG3C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:50.146" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:20.347" grpc.time_ms=0.658 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240gitlab # [ 270.153875] gitaly[911]: time="2026-11-10T09:54:20.378Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2HGK6J1DW2JC40C3TG3C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T09:54:30.281" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T09:54:20.372" grpc.time_ms=6.481 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2241gitlab # [ 270.196672] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1132242gitlab # [ 270.204326] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bf3d502c2e1e490241e098b6","created_at":"2026-11-10T09:54:19.643Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a9eedf03e9fb3b8e56ef1fc942189a2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.658Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bf3d502c2e1e490241e098b6: done: 0.306266 sec","job_status":"done","queue_duration_s":0.463879,"scheduling_latency_s":0.46392,"redis_calls":564,"redis_duration_s":0.101939,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.100548,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.227651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.306266,"completed_at":"2026-11-10T09:54:20.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2243gitlab # [ 270.242201] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"69641d911a9aa2981b70e7c0","created_at":"2026-11-10T09:54:19.666Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19c856086abec572bb4978f2b98e99a8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.677Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-69641d911a9aa2981b70e7c0: start","job_status":"start","queue_duration_s":0.789233,"scheduling_latency_s":0.789273}2244gitlab # [ 270.444729] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"69641d911a9aa2981b70e7c0","created_at":"2026-11-10T09:54:19.666Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19c856086abec572bb4978f2b98e99a8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.677Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-69641d911a9aa2981b70e7c0: done: 0.202573 sec","job_status":"done","queue_duration_s":0.789233,"scheduling_latency_s":0.789273,"redis_calls":13,"redis_duration_s":0.002881,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.168568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001011022,"exclusive_lock_hold_duration_s":0.18009328,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.202573,"completed_at":"2026-11-10T09:54:20.669Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 270.446556] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e6a79187860ff4458bc41a27","created_at":"2026-11-10T09:54:19.722Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f705550f6b3b8b047c27a36623e8a9d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.736Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e6a79187860ff4458bc41a27: start","job_status":"start","queue_duration_s":0.934394,"scheduling_latency_s":0.934418}2246gitlab # [ 270.505911] nixbot[2700]: {"ts": "2026-11-10T09:54:20.730Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "a59288aee385d50e3b1a10cc2d2c3866b2c77c0a", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}2247gitlab # [ 270.535107] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"e6a79187860ff4458bc41a27","created_at":"2026-11-10T09:54:19.722Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f705550f6b3b8b047c27a36623e8a9d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.736Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-e6a79187860ff4458bc41a27: done: 0.088151 sec","job_status":"done","queue_duration_s":0.934394,"scheduling_latency_s":0.934418,"redis_calls":11,"redis_duration_s":0.002872,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000505371,"exclusive_lock_hold_duration_s":0.014830377,"duration_s":0.088151,"completed_at":"2026-11-10T09:54:20.758Z","load_balancing_strategy":"primary","db_duration_s":0.046083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 270.538838] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.763Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"77a92cce7a63374c9eb35736","created_at":"2026-11-10T09:54:19.745Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc9599c2746657855008c368645ad8c6","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.760Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-77a92cce7a63374c9eb35736: start","job_status":"start","queue_duration_s":1.002719,"scheduling_latency_s":1.002753}2249gitlab # [ 270.559317] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.783Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"77a92cce7a63374c9eb35736","created_at":"2026-11-10T09:54:19.745Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc9599c2746657855008c368645ad8c6","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.760Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-77a92cce7a63374c9eb35736: done: 0.02043 sec","job_status":"done","queue_duration_s":1.002719,"scheduling_latency_s":1.002753,"redis_calls":11,"redis_duration_s":0.0018909999999999999,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000317,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000298,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02043,"completed_at":"2026-11-10T09:54:20.783Z","load_balancing_strategy":"primary","db_duration_s":0.000629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250gitlab # [ 270.614197] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.785Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"6e0ddb7f37d0071a2a37cec0","created_at":"2026-11-10T09:54:19.774Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6179230741098820a9a94f104d6d2eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.783Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-6e0ddb7f37d0071a2a37cec0: start","job_status":"start","queue_duration_s":1.001429,"scheduling_latency_s":1.00145}2251gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses2252gitlab # [ 270.614860] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.793Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2253gitlab # [ 270.615464] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"6e0ddb7f37d0071a2a37cec0","created_at":"2026-11-10T09:54:19.774Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6179230741098820a9a94f104d6d2eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.783Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-6e0ddb7f37d0071a2a37cec0: done: 0.013667 sec","job_status":"done","queue_duration_s":1.001429,"scheduling_latency_s":1.00145,"redis_calls":9,"redis_duration_s":0.001256,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013667,"completed_at":"2026-11-10T09:54:20.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254gitlab # [ 270.632103] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d9db02c34743a784cf03ee7e","created_at":"2026-11-10T09:54:19.791Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c746242fb3b7af08c57a61eb95c0d84","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T09:54:19.804Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d9db02c34743a784cf03ee7e: start","job_status":"start","queue_duration_s":0.995628,"scheduling_latency_s":0.995645}2255gitlab # [ 270.642947] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.807Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2256gitlab # [ 270.645819] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d9db02c34743a784cf03ee7e","created_at":"2026-11-10T09:54:19.791Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c746242fb3b7af08c57a61eb95c0d84","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T09:54:19.804Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d9db02c34743a784cf03ee7e: done: 0.040889 sec","job_status":"done","queue_duration_s":0.995628,"scheduling_latency_s":0.995645,"redis_calls":9,"redis_duration_s":0.001467,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040889,"completed_at":"2026-11-10T09:54:20.840Z","load_balancing_strategy":"primary","db_duration_s":0.000367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 270.681374] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b60141247843d7a71ab0cb08","created_at":"2026-11-10T09:54:19.819Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf6f7208b3afcf2289e5c6a4e92c2bd7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.835Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b60141247843d7a71ab0cb08: start","job_status":"start","queue_duration_s":1.006759,"scheduling_latency_s":1.006778}2258gitlab # [ 270.731218] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.871Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b60141247843d7a71ab0cb08","created_at":"2026-11-10T09:54:19.819Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf6f7208b3afcf2289e5c6a4e92c2bd7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.835Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b60141247843d7a71ab0cb08: done: 0.029163 sec","job_status":"done","queue_duration_s":1.006759,"scheduling_latency_s":1.006778,"redis_calls":11,"redis_duration_s":0.001944,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000839,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000384127,"exclusive_lock_hold_duration_s":0.005683403,"duration_s":0.029163,"completed_at":"2026-11-10T09:54:20.871Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259gitlab # [ 270.732644] gitaly[911]: time="2026-11-10T09:54:20.894Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2HYR4M4DR6KF6MEJA5YW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:20.883" grpc.time_ms=10.858 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2260gitlab # [ 270.743563] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1342261gitlab # [ 270.744158] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"202b3072051f8669d5ec2110","created_at":"2026-11-10T09:54:19.845Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e6ff6eb9db501a9067efb6e74e82c41","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.858Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-202b3072051f8669d5ec2110: start","job_status":"start","queue_duration_s":1.013951,"scheduling_latency_s":1.013981}2262gitlab # [ 270.755611] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"202b3072051f8669d5ec2110","created_at":"2026-11-10T09:54:19.845Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e6ff6eb9db501a9067efb6e74e82c41","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.858Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-202b3072051f8669d5ec2110: done: 0.026531 sec","job_status":"done","queue_duration_s":1.013951,"scheduling_latency_s":1.013981,"redis_calls":11,"redis_duration_s":0.002844,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001085,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000582477,"exclusive_lock_hold_duration_s":0.003960559,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026531,"completed_at":"2026-11-10T09:54:20.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab # [ 270.792762] gitaly[911]: time="2026-11-10T09:54:20.962Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2J3MG078KD8XVRZECX6D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T09:54:50.761" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T09:54:20.961" grpc.time_ms=0.599 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2264gitlab # [ 270.803644] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 712265gitlab # [ 270.805830] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fc05d2fe4417fb44a24a9569","created_at":"2026-11-10T09:54:19.873Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"602eebb1182da6c274518db2056b5d8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T09:54:19.884Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fc05d2fe4417fb44a24a9569: start","job_status":"start","queue_duration_s":1.016344,"scheduling_latency_s":1.016372}2266gitlab # [ 270.854146] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.913Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2267gitlab # [ 270.854929] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fc05d2fe4417fb44a24a9569","created_at":"2026-11-10T09:54:19.873Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"602eebb1182da6c274518db2056b5d8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T09:54:19.884Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fc05d2fe4417fb44a24a9569: done: 0.023156 sec","job_status":"done","queue_duration_s":1.016344,"scheduling_latency_s":1.016372,"redis_calls":9,"redis_duration_s":0.00183,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000812,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023156,"completed_at":"2026-11-10T09:54:20.923Z","load_balancing_strategy":"primary","db_duration_s":0.0007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268gitlab # [ 270.866978] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.925Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"37843c801934b36cb269fda8","created_at":"2026-11-10T09:54:19.894Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7d77ad4d5c854792c22d155708e114d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.903Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-37843c801934b36cb269fda8: start","job_status":"start","queue_duration_s":1.021926,"scheduling_latency_s":1.021952}2269gitlab # [ 270.928849] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"37843c801934b36cb269fda8","created_at":"2026-11-10T09:54:19.894Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7d77ad4d5c854792c22d155708e114d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.903Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-37843c801934b36cb269fda8: done: 0.035296 sec","job_status":"done","queue_duration_s":1.021926,"scheduling_latency_s":1.021952,"redis_calls":9,"redis_duration_s":0.001797,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035296,"completed_at":"2026-11-10T09:54:20.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005391,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2270gitlab # [ 270.938389] gitlab-workhorse[825]: time="2026-11-10T09:54:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672271gitlab # [ 270.941026] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1252272gitlab # [ 270.943633] gitaly[911]: time="2026-11-10T09:54:21.043Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2J3PGN4J653N6GAQY693 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.035" pid=911 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2273gitlab # [ 270.958207] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.962Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5f2d3845720dd37855abd143","created_at":"2026-11-10T09:54:19.929Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fdc48478e62008d1ff45661106f82207","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.946Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5f2d3845720dd37855abd143: start","job_status":"start","queue_duration_s":1.016125,"scheduling_latency_s":1.01616}2274gitlab # [ 270.976890] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.991Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5f2d3845720dd37855abd143","created_at":"2026-11-10T09:54:19.929Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fdc48478e62008d1ff45661106f82207","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.946Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5f2d3845720dd37855abd143: done: 0.029589 sec","job_status":"done","queue_duration_s":1.016125,"scheduling_latency_s":1.01616,"redis_calls":9,"redis_duration_s":0.0029,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002289,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029589,"completed_at":"2026-11-10T09:54:20.991Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2275gitlab # [ 271.037420] gitaly[911]: time="2026-11-10T09:54:21.043Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2J3PGN4J653N6GAQY693 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.035" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2276gitlab # [ 271.050131] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:20.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b75d96a1b65d5653ccea693f","created_at":"2026-11-10T09:54:19.960Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0bdc7750f550912139cc1892f02552a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.970Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b75d96a1b65d5653ccea693f: start","job_status":"start","queue_duration_s":1.022322,"scheduling_latency_s":1.02235}2277gitlab # [ 271.097409] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.003Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2278gitlab # [ 271.098104] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b75d96a1b65d5653ccea693f","created_at":"2026-11-10T09:54:19.960Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0bdc7750f550912139cc1892f02552a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:19.970Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b75d96a1b65d5653ccea693f: done: 0.016981 sec","job_status":"done","queue_duration_s":1.022322,"scheduling_latency_s":1.02235,"redis_calls":9,"redis_duration_s":0.001268,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016981,"completed_at":"2026-11-10T09:54:21.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 271.108795] gitaly[911]: time="2026-11-10T09:54:21.043Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2J3PGN4J653N6GAQY693 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.035" grpc.time_ms=7.794 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2280gitlab # [ 271.159732] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"36f3223906e4587c26b05fb3","created_at":"2026-11-10T09:54:19.987Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a96e5c5fe1f64ba9ec1ca2b29eb3df9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.000Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-36f3223906e4587c26b05fb3: start","job_status":"start","queue_duration_s":1.011319,"scheduling_latency_s":1.011342}2281gitlab # [ 271.160605] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.028Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"36f3223906e4587c26b05fb3","created_at":"2026-11-10T09:54:19.987Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a96e5c5fe1f64ba9ec1ca2b29eb3df9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.000Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-36f3223906e4587c26b05fb3: done: 0.016873 sec","job_status":"done","queue_duration_s":1.011319,"scheduling_latency_s":1.011342,"redis_calls":9,"redis_duration_s":0.001258,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016873,"completed_at":"2026-11-10T09:54:21.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2282gitlab # [ 271.204665] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 652283gitlab # [ 271.205604] gitaly[911]: time="2026-11-10T09:54:21.373Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72E2JFWBH6FPGHTM58AYJX1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.362" grpc.time_ms=11.036 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2284gitlab # [ 271.206752] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0524446b1ab1260d814670ab","created_at":"2026-11-10T09:54:20.009Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5136c9e306800d37322183c6fbc3161e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T09:54:20.016Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0524446b1ab1260d814670ab: start","job_status":"start","queue_duration_s":1.012692,"scheduling_latency_s":1.012713}2285gitlab # [ 271.207647] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.057Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0524446b1ab1260d814670ab","created_at":"2026-11-10T09:54:20.009Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-6c9891f6b38a488c","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5136c9e306800d37322183c6fbc3161e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T09:54:20.016Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0524446b1ab1260d814670ab: done: 0.028138 sec","job_status":"done","queue_duration_s":1.012692,"scheduling_latency_s":1.012713,"redis_calls":9,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00031,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028138,"completed_at":"2026-11-10T09:54:21.057Z","load_balancing_strategy":"primary","db_duration_s":0.003095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2286gitlab # [ 271.251962] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e33e284ff748d4898084ec6","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e33e284ff748d4898084ec6: start","job_status":"start","queue_duration_s":1.032404,"scheduling_latency_s":1.032431}2287gitlab # [ 271.252645] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e33e284ff748d4898084ec6","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e33e284ff748d4898084ec6: done: 0.022205 sec","job_status":"done","queue_duration_s":1.032404,"scheduling_latency_s":1.032431,"redis_calls":15,"redis_duration_s":0.002138,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00031,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022205,"completed_at":"2026-11-10T09:54:21.081Z","load_balancing_strategy":"primary","db_duration_s":0.000397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2288gitlab # [ 271.291120] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7336f4842a43d7cade26436a","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7336f4842a43d7cade26436a: start","job_status":"start","queue_duration_s":1.056604,"scheduling_latency_s":1.056636}2289gitlab # [ 271.330869] gitaly[911]: time="2026-11-10T09:54:21.433Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72E2JHZF3A5MZ1NCFRA915W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.425" pid=911 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc2290gitlab # [ 271.331634] gitlab-workhorse[825]: time="2026-11-10T09:54:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672291gitlab # [ 271.332073] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:09:54:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 582292gitlab # [ 271.332618] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7336f4842a43d7cade26436a","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7336f4842a43d7cade26436a: done: 0.015471 sec","job_status":"done","queue_duration_s":1.056604,"scheduling_latency_s":1.056636,"redis_calls":13,"redis_duration_s":0.002151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015471,"completed_at":"2026-11-10T09:54:21.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293gitlab # [ 271.386941] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c240c781bad98daa6020da23","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c240c781bad98daa6020da23: start","job_status":"start","queue_duration_s":1.073411,"scheduling_latency_s":1.073432}2294gitlab # [ 271.387960] gitaly[911]: time="2026-11-10T09:54:21.433Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2JHZF3A5MZ1NCFRA915W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.425" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2295gitlab # [ 271.388844] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c240c781bad98daa6020da23","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c240c781bad98daa6020da23: done: 0.013059 sec","job_status":"done","queue_duration_s":1.073411,"scheduling_latency_s":1.073432,"redis_calls":13,"redis_duration_s":0.00193,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013059,"completed_at":"2026-11-10T09:54:21.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 271.410783] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61744b37bb96ae12147ddc80","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61744b37bb96ae12147ddc80: start","job_status":"start","queue_duration_s":1.087649,"scheduling_latency_s":1.087668}2297gitlab # [ 271.454395] gitaly[911]: time="2026-11-10T09:54:21.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72E2JHZF3A5MZ1NCFRA915W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T09:54:21.425" grpc.time_ms=8.358 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=911 remote_ip="2001:db8:1::1" span.kind=server system=grpc2298gitlab # [ 271.455451] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61744b37bb96ae12147ddc80","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61744b37bb96ae12147ddc80: done: 0.011613 sec","job_status":"done","queue_duration_s":1.087649,"scheduling_latency_s":1.087668,"redis_calls":13,"redis_duration_s":0.001824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001253,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011613,"completed_at":"2026-11-10T09:54:21.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2299gitlab # [ 271.512218] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cd5d06d13bdbd4a6788a0a2","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cd5d06d13bdbd4a6788a0a2: start","job_status":"start","queue_duration_s":1.100325,"scheduling_latency_s":1.100342}2300gitlab # [ 271.513050] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cd5d06d13bdbd4a6788a0a2","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cd5d06d13bdbd4a6788a0a2: done: 0.011772 sec","job_status":"done","queue_duration_s":1.100325,"scheduling_latency_s":1.100342,"redis_calls":13,"redis_duration_s":0.001977,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011772,"completed_at":"2026-11-10T09:54:21.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2301gitlab # [ 271.562379] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b72cdd0529c235f4412ad54e","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b72cdd0529c235f4412ad54e: start","job_status":"start","queue_duration_s":1.113204,"scheduling_latency_s":1.113223}2302gitlab # [ 271.563280] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b72cdd0529c235f4412ad54e","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b72cdd0529c235f4412ad54e: done: 0.011846 sec","job_status":"done","queue_duration_s":1.113204,"scheduling_latency_s":1.113223,"redis_calls":13,"redis_duration_s":0.002014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011846,"completed_at":"2026-11-10T09:54:21.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2303gitlab # [ 271.578662] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efc6f99a1a5059af5c2bf445","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efc6f99a1a5059af5c2bf445: start","job_status":"start","queue_duration_s":1.126113,"scheduling_latency_s":1.12613}2304gitlab # [ 271.590660] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efc6f99a1a5059af5c2bf445","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efc6f99a1a5059af5c2bf445: done: 0.011962 sec","job_status":"done","queue_duration_s":1.126113,"scheduling_latency_s":1.12613,"redis_calls":13,"redis_duration_s":0.001983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011962,"completed_at":"2026-11-10T09:54:21.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2305gitlab # [ 271.645142] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f69c42bdc10fbc02e5c9a6f","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f69c42bdc10fbc02e5c9a6f: start","job_status":"start","queue_duration_s":1.139654,"scheduling_latency_s":1.139672}2306gitlab # [ 271.645923] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f69c42bdc10fbc02e5c9a6f","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f69c42bdc10fbc02e5c9a6f: done: 0.011822 sec","job_status":"done","queue_duration_s":1.139654,"scheduling_latency_s":1.139672,"redis_calls":13,"redis_duration_s":0.002071,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001447,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011822,"completed_at":"2026-11-10T09:54:21.177Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2307gitlab # [ 271.695906] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"972de5d1da1e95d25fb98d69","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-972de5d1da1e95d25fb98d69: start","job_status":"start","queue_duration_s":1.152608,"scheduling_latency_s":1.152626}2308gitlab # [ 271.696680] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"972de5d1da1e95d25fb98d69","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-972de5d1da1e95d25fb98d69: done: 0.01167 sec","job_status":"done","queue_duration_s":1.152608,"scheduling_latency_s":1.152626,"redis_calls":13,"redis_duration_s":0.0019950000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01167,"completed_at":"2026-11-10T09:54:21.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2309gitlab # [ 271.747161] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d152792c4817cbbf3270df0","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d152792c4817cbbf3270df0: start","job_status":"start","queue_duration_s":1.165471,"scheduling_latency_s":1.165488}2310gitlab # [ 271.747838] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d152792c4817cbbf3270df0","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d152792c4817cbbf3270df0: done: 0.011607 sec","job_status":"done","queue_duration_s":1.165471,"scheduling_latency_s":1.165488,"redis_calls":13,"redis_duration_s":0.001937,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011607,"completed_at":"2026-11-10T09:54:21.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab # [ 271.797398] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c6f564988bcd037ab8335b6","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c6f564988bcd037ab8335b6: start","job_status":"start","queue_duration_s":1.178143,"scheduling_latency_s":1.178159}2312gitlab # [ 271.798259] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c6f564988bcd037ab8335b6","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c6f564988bcd037ab8335b6: done: 0.011841 sec","job_status":"done","queue_duration_s":1.178143,"scheduling_latency_s":1.178159,"redis_calls":13,"redis_duration_s":0.002203,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001574,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011841,"completed_at":"2026-11-10T09:54:21.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 271.845622] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a2935c43b9c367928c68461","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a2935c43b9c367928c68461: start","job_status":"start","queue_duration_s":1.19103,"scheduling_latency_s":1.191057}2314gitlab # [ 271.846800] gitaly[911]: time="2026-11-10T09:54:22.013Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2H7MR6JEE0397YZGKNXJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:22.011" grpc.time_ms=1.962 pid=911 span.kind=server system=grpc2315gitlab # [ 271.847318] gitaly[911]: time="2026-11-10T09:54:22.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2H7MR6JEE0397YZGKNXJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:22.035" grpc.time_ms=0.394 pid=911 span.kind=server system=grpc2316gitlab # [ 271.847945] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a2935c43b9c367928c68461","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a2935c43b9c367928c68461: done: 0.011628 sec","job_status":"done","queue_duration_s":1.19103,"scheduling_latency_s":1.191057,"redis_calls":13,"redis_duration_s":0.002051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001377,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011628,"completed_at":"2026-11-10T09:54:21.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2317gitlab # [ 271.901934] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edf60982cb15fa51ab0e12ed","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edf60982cb15fa51ab0e12ed: start","job_status":"start","queue_duration_s":1.203815,"scheduling_latency_s":1.203831}2318gitlab # [ 271.902729] gitaly[911]: time="2026-11-10T09:54:22.056Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72E2H7MR6JEE0397YZGKNXJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T09:54:22.056" grpc.time_ms=0.267 pid=911 span.kind=server system=grpc2319gitlab # [ 271.903447] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edf60982cb15fa51ab0e12ed","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edf60982cb15fa51ab0e12ed: done: 0.011556 sec","job_status":"done","queue_duration_s":1.203815,"scheduling_latency_s":1.203831,"redis_calls":13,"redis_duration_s":0.001983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011556,"completed_at":"2026-11-10T09:54:21.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2320gitlab # [ 271.973426] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6748d97e480589b3dca1afee","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6748d97e480589b3dca1afee: start","job_status":"start","queue_duration_s":1.216414,"scheduling_latency_s":1.21643}2321gitlab # [ 271.974195] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6748d97e480589b3dca1afee","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6748d97e480589b3dca1afee: done: 0.011164 sec","job_status":"done","queue_duration_s":1.216414,"scheduling_latency_s":1.21643,"redis_calls":13,"redis_duration_s":0.001939,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001294,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011164,"completed_at":"2026-11-10T09:54:21.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2322gitlab # [ 272.015131] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bb9fb379e95dd5e04a9f829","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bb9fb379e95dd5e04a9f829: start","job_status":"start","queue_duration_s":1.228622,"scheduling_latency_s":1.228638}2323gitlab # [ 272.015781] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bb9fb379e95dd5e04a9f829","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bb9fb379e95dd5e04a9f829: done: 0.018064 sec","job_status":"done","queue_duration_s":1.228622,"scheduling_latency_s":1.228638,"redis_calls":13,"redis_duration_s":0.00312,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001838,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018064,"completed_at":"2026-11-10T09:54:21.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2324gitlab # [ 272.028521] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6ab99f66be1ba394989d4df","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6ab99f66be1ba394989d4df: start","job_status":"start","queue_duration_s":1.248079,"scheduling_latency_s":1.248108}2325gitlab # [ 272.039249] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6ab99f66be1ba394989d4df","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6ab99f66be1ba394989d4df: done: 0.012962 sec","job_status":"done","queue_duration_s":1.248079,"scheduling_latency_s":1.248108,"redis_calls":13,"redis_duration_s":0.0018729999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012962,"completed_at":"2026-11-10T09:54:21.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2326gitlab # [ 272.077223] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9645b61d3f2ba7e2a5bf377e","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9645b61d3f2ba7e2a5bf377e: start","job_status":"start","queue_duration_s":1.262171,"scheduling_latency_s":1.262189}2327gitlab # [ 272.083275] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9645b61d3f2ba7e2a5bf377e","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9645b61d3f2ba7e2a5bf377e: done: 0.011234 sec","job_status":"done","queue_duration_s":1.262171,"scheduling_latency_s":1.262189,"redis_calls":13,"redis_duration_s":0.001823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011234,"completed_at":"2026-11-10T09:54:21.299Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2328gitlab # [ 272.124614] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"090b29cf4a2b7bb9a9e10a86","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-090b29cf4a2b7bb9a9e10a86: start","job_status":"start","queue_duration_s":1.274394,"scheduling_latency_s":1.274413}2329gitlab # [ 272.127499] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"090b29cf4a2b7bb9a9e10a86","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-090b29cf4a2b7bb9a9e10a86: done: 0.012325 sec","job_status":"done","queue_duration_s":1.274394,"scheduling_latency_s":1.274413,"redis_calls":13,"redis_duration_s":0.002741,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012325,"completed_at":"2026-11-10T09:54:21.313Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2330gitlab # [ 272.167825] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c731f664975ffd0a9ae05c10","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c731f664975ffd0a9ae05c10: start","job_status":"start","queue_duration_s":1.287905,"scheduling_latency_s":1.287922}2331gitlab # [ 272.171814] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c731f664975ffd0a9ae05c10","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c731f664975ffd0a9ae05c10: done: 0.010533 sec","job_status":"done","queue_duration_s":1.287905,"scheduling_latency_s":1.287922,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010533,"completed_at":"2026-11-10T09:54:21.324Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2332gitlab # [ 272.209675] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e895c66da99154253ee5778","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e895c66da99154253ee5778: start","job_status":"start","queue_duration_s":1.299497,"scheduling_latency_s":1.299515}2333gitlab # [ 272.249648] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e895c66da99154253ee5778","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e895c66da99154253ee5778: done: 0.010615 sec","job_status":"done","queue_duration_s":1.299497,"scheduling_latency_s":1.299515,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010615,"completed_at":"2026-11-10T09:54:21.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2334gitlab # [ 272.250372] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1dbc7a8c5c31fc4ec32f7eab","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1dbc7a8c5c31fc4ec32f7eab: start","job_status":"start","queue_duration_s":1.31105,"scheduling_latency_s":1.311066}2335gitlab # [ 272.288986] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1dbc7a8c5c31fc4ec32f7eab","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1dbc7a8c5c31fc4ec32f7eab: done: 0.011429 sec","job_status":"done","queue_duration_s":1.31105,"scheduling_latency_s":1.311066,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011429,"completed_at":"2026-11-10T09:54:21.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2336gitlab # [ 272.329705] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0b908b60667f11b2fa1629c","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0b908b60667f11b2fa1629c: start","job_status":"start","queue_duration_s":1.323417,"scheduling_latency_s":1.323434}2337gitlab # [ 272.330782] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0b908b60667f11b2fa1629c","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0b908b60667f11b2fa1629c: done: 0.017399 sec","job_status":"done","queue_duration_s":1.323417,"scheduling_latency_s":1.323434,"redis_calls":13,"redis_duration_s":0.0038929999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002404,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001489,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017399,"completed_at":"2026-11-10T09:54:21.367Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2338gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a59288aee385d50e3b1a10cc2d2c3866b2c77c0a/statuses, in 1.76 seconds)2339gitlab # [ 272.342559] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2fd1e13881fd35833b91c79","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2fd1e13881fd35833b91c79: start","job_status":"start","queue_duration_s":1.342164,"scheduling_latency_s":1.342192}2340(finished: subtest: eval and build post commit statuses, in 8.73 seconds)2341(finished: run the VM test script, in 332.94 seconds)2342test script finished in 332.96s2343cleanup2344kill QemuMachine (pid 46)2345gitlab # [ 272.385784] sidekiq-cluster[2576]: {"severity":"INFO","time":"2026-11-10T09:54:21.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2fd1e13881fd35833b91c79","created_at":"2026-11-10T09:54:19.798Z","trace_propagation_headers":{"sentry-trace":"7aedfe394d604bedb059e62cea49355a-b02bcbd1c557438f","baggage":"sentry-trace_id=7aedfe394d604bedb059e62cea49355a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d032788a61acd0342913e5f518770b96","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T09:54:20.026Z","job_size_bytes":2,"pid":2576,"sidekiq_tid":"m81w","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2fd1e13881fd35833b91c79: done: 0.016079 sec","job_status":"done","queue_duration_s":1.342164,"scheduling_latency_s":1.342192,"redis_calls":13,"redis_duration_s":0.00213,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2346gitlab # 02347(finished: cleanup, in 1.80 seconds)2348additionally exposed symbols:2349 gitlab,2350 vlan1,2351 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2352[]2353[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-10T09:53:21.418Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2354[]2355[]2356[]2357[]2358[]2359[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-10T09:54:13.744347Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2360[]2361[]2362[{'id': 5, 'sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T09:54:15.560Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2363[{'id': 5, 'sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T09:54:15.560Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2364[{'id': 5, 'sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T09:54:15.560Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2365[{'id': 5, 'sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T09:54:15.560Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2366[{'id': 5, 'sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-11-10T09:54:15.560Z', 'started_at': None, 'finished_at': '2026-11-10T09:54:20.130Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'a59288aee385d50e3b1a10cc2d2c3866b2c77c0a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-11-10T09:54:20.273Z', 'started_at': None, 'finished_at': '2026-11-10T09:54:20.397Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]