vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #354
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.zaO6DzFX2F', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: c5e01e52-77ce-47c2-a1c3-6717f911954016gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618gitlab # 19gitlab # Allocating group tables: 0/8 done20gitlab # Writing inode tables: 0/8 done21gitlab # Creating journal (8192 blocks): done22gitlab # Writing superblocks and filesystem accounting information: 0/8 done23gitlab # 24gitlab # Virtualisation disk image created.25gitlab # Creating Nix store image...26gitlab # Created Nix store image.27(finished: start all VMs, in 54.99 seconds)28gitlab: waiting for unit setup-gitlab.service29gitlab: waiting for the VM to finish booting30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6j8s7qi3n0s19hl82zlq8xpd9vb568i4-nixos-system-gitlab-test/init regInfo=/nix/store/gk8br0h5cicccnxvh4krama90x0lw0af-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 509987511 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000808] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000834] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000875] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002727] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002738] Using GB pages for direct mapping73gitlab # [ 0.002815] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002823] ACPI: Early table checksum verification disabled75gitlab # [ 0.002826] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002830] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002834] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002841] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002843] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002845] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002847] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002848] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002849] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002850] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002851] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002851] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002852] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002852] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003072] No NUMA configuration found90gitlab # [ 0.003073] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003076] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.005429] Zone ranges:93gitlab # [ 0.005430] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005431] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005432] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005433] Device empty97gitlab # [ 0.005434] Movable zone start for each node98gitlab # [ 0.005435] Early memory node ranges99gitlab # [ 0.005435] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005436] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005437] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005438] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005459] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005737] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.116532] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.117025] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.117041] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.117078] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.117082] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.117084] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.117085] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.117086] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.117087] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.117090] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.117091] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.117096] TSC deadline timer available117gitlab # [ 0.117101] CPU topo: Max. logical packages: 1118gitlab # [ 0.117102] CPU topo: Max. logical dies: 1119gitlab # [ 0.117102] CPU topo: Max. dies per package: 1120gitlab # [ 0.117106] CPU topo: Max. threads per core: 1121gitlab # [ 0.117106] CPU topo: Num. cores per package: 4122gitlab # [ 0.117106] CPU topo: Num. threads per package: 4123gitlab # [ 0.117107] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.117133] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.117151] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.117154] kvm-guest: setup PV sched yield127gitlab # [ 0.117172] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.117174] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.117175] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.117177] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.117179] Booting paravirtualized kernel on KVM132gitlab # [ 0.117183] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.121650] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.124176] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.124228] kvm-guest: PV spinlocks enabled136gitlab # [ 0.124230] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.124233] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6j8s7qi3n0s19hl82zlq8xpd9vb568i4-nixos-system-gitlab-test/init regInfo=/nix/store/gk8br0h5cicccnxvh4krama90x0lw0af-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.124333] Unknown kernel command line parameters "regInfo=/nix/store/gk8br0h5cicccnxvh4krama90x0lw0af-closure-info/registration", will be passed to user space.139gitlab # [ 0.124346] random: crng init done140gitlab # [ 0.124347] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.133432] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.137835] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.137915] software IO TLB: area num 4.144gitlab # [ 0.214875] Fallback order for Node 0: 0145gitlab # [ 0.214883] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.214885] Policy zone: Normal147gitlab # [ 0.217544] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.227896] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.241442] allocated 12582912 bytes of page_ext150gitlab # [ 0.251757] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.251759] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.252628] Dynamic Preempt: lazy153gitlab # [ 0.252833] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.252833] rcu: RCU event tracing is enabled.155gitlab # [ 0.252834] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.252836] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.252836] Rude variant of Tasks RCU enabled.158gitlab # [ 0.252837] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.252837] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.252838] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.252859] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.252861] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.252861] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.257433] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.257728] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.257737] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.257849] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.261451] Console: colour VGA+ 80x25169gitlab # [ 0.261454] printk: legacy console [tty0] enabled170gitlab # [ 0.291671] printk: legacy console [ttyS0] enabled171gitlab # [ 0.398983] ACPI: Core revision 20250807172gitlab # [ 0.399927] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.401552] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.402613] x2apic enabled175gitlab # [ 0.403358] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.404329] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.405626] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.407127] kvm-guest: setup PV IPIs179gitlab # [ 0.408752] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.409824] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.411625] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.412710] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.413745] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.415622] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.416626] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.417622] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.418622] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.420410] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.421379] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.422427] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.423630] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.424623] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.425623] active return thunk: srso_alias_return_thunk194gitlab # [ 0.426643] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.427622] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.429430] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.430622] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.431622] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.432622] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.433622] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.434622] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.435622] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.436622] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.438427] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.439417] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.440419] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.470601] Freeing SMP alternatives memory: 44K208gitlab # [ 0.471243] pid_max: default: 32768 minimum: 301209gitlab # [ 0.472317] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.472720] landlock: Up and running.211gitlab # [ 0.474098] Yama: becoming mindful.212gitlab # [ 0.474835] LSM support for eBPF active213gitlab # [ 0.475842] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.476772] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.480032] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.481207] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.481633] ... version: 2218gitlab # [ 0.482404] ... bit width: 48219gitlab # [ 0.482624] ... generic counters: 6220gitlab # [ 0.483391] ... generic bitmap: 000000000000003f221gitlab # [ 0.483624] ... fixed-purpose counters: 0222gitlab # [ 0.484383] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.484624] ... value mask: 0000ffffffffffff224gitlab # [ 0.485581] ... max period: 00007fffffffffff225gitlab # [ 0.486381] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.486729] signal: max sigframe size: 3376227gitlab # [ 0.487609] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.488281] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.488812] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.494355] smp: Bringing up secondary CPUs ...231gitlab # [ 0.495014] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.495633] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.498813] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.500397] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.500955] Memory: 6011564K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.502625] devtmpfs: initialized237gitlab # [ 0.503686] x86/mm: Memory block size: 128MB238gitlab # [ 0.506779] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.507706] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.508727] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.509945] PM: RTC time: 04:22:56, date: 2026-09-08242gitlab # [ 0.513552] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.515755] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.516688] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.518868] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.519636] audit: initializing netlink subsys (disabled)247gitlab # [ 0.521679] audit: type=2000 audit(1788841376.860:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.521925] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.523625] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.524625] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.525624] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.526624] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.527640] cpuidle: using governor menu254gitlab # [ 0.529846] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.530883] PCI: Using configuration type 1 for base access256gitlab # [ 0.531625] PCI: Using configuration type 1 for extended access257gitlab # [ 0.532781] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.535673] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.536626] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.537625] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.538624] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.543598] ACPI: Added _OSI(Module Device)263gitlab # [ 0.544633] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.546631] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.554571] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.560904] ACPI: Interpreter enabled267gitlab # [ 0.561643] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.562375] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.562888] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.566632] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.569201] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.582352] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.585640] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.589922] acpiphp: Slot [3] registered275gitlab # [ 0.591740] acpiphp: Slot [4] registered276gitlab # [ 0.593735] acpiphp: Slot [5] registered277gitlab # [ 0.595754] acpiphp: Slot [6] registered278gitlab # [ 0.596715] acpiphp: Slot [7] registered279gitlab # [ 0.598705] acpiphp: Slot [8] registered280gitlab # [ 0.599697] acpiphp: Slot [9] registered281gitlab # [ 0.601716] acpiphp: Slot [10] registered282gitlab # [ 0.602707] acpiphp: Slot [11] registered283gitlab # [ 0.603745] acpiphp: Slot [12] registered284gitlab # [ 0.605713] acpiphp: Slot [13] registered285gitlab # [ 0.606670] acpiphp: Slot [14] registered286gitlab # [ 0.608668] acpiphp: Slot [15] registered287gitlab # [ 0.609666] acpiphp: Slot [16] registered288gitlab # [ 0.610692] acpiphp: Slot [17] registered289gitlab # [ 0.612668] acpiphp: Slot [18] registered290gitlab # [ 0.613665] acpiphp: Slot [19] registered291gitlab # [ 0.615666] acpiphp: Slot [20] registered292gitlab # [ 0.616611] acpiphp: Slot [21] registered293gitlab # [ 0.617117] acpiphp: Slot [22] registered294gitlab # [ 0.617644] acpiphp: Slot [23] registered295gitlab # [ 0.618369] acpiphp: Slot [24] registered296gitlab # [ 0.618654] acpiphp: Slot [25] registered297gitlab # [ 0.619364] acpiphp: Slot [26] registered298gitlab # [ 0.620646] acpiphp: Slot [27] registered299gitlab # [ 0.621421] acpiphp: Slot [28] registered300gitlab # [ 0.621646] acpiphp: Slot [29] registered301gitlab # [ 0.622356] acpiphp: Slot [30] registered302gitlab # [ 0.623644] acpiphp: Slot [31] registered303gitlab # [ 0.624397] PCI host bridge to bus 0000:00304gitlab # [ 0.624634] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.625625] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.626625] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.628634] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.631636] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.634632] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.637455] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.641816] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.646731] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.653719] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.656700] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.658630] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.661630] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.664630] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.667094] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.674297] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.679422] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.683190] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.685642] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.688061] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.702720] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.704639] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.705646] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.707074] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.709633] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.714663] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.715559] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.716443] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.730632] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.733223] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.738665] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.739589] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.740442] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.743931] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.748661] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.749586] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.750437] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.752887] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.756701] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.758632] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.759646] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.762860] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.766663] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.767578] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.768660] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.772052] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.776661] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.777553] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.778405] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.781883] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.786663] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.787586] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.788440] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.789610] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.792949] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.795638] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.796677] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.798913] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.803484] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.804379] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.805647] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.807909] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.811632] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.813632] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.814646] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.820253] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.820800] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.821784] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.823755] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.824716] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.828217] iommu: Default domain type: Translated375gitlab # [ 0.828258] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.829917] ACPI: bus type USB registered377gitlab # [ 0.830697] usbcore: registered new interface driver usbfs378gitlab # [ 0.831639] usbcore: registered new interface driver hub379gitlab # [ 0.832693] usbcore: registered new device driver usb380gitlab # [ 0.835272] NetLabel: Initializing381gitlab # [ 0.836627] NetLabel: domain hash size = 128382gitlab # [ 0.837421] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.838471] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.839389] PCI: Using ACPI for IRQ routing385gitlab # [ 0.840642] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.841621] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.841621] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.843628] vgaarb: loaded389gitlab # [ 0.844363] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.845625] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.851770] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.854909] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.855631] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.857166] pnp: PnP ACPI init395gitlab # [ 0.858319] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.867806] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.869394] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.870434] NET: Registered PF_INET protocol family399gitlab # [ 0.872625] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.890639] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.892217] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.893634] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.897609] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.899005] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.900576] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.902267] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.903760] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.905214] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.906231] NET: Registered PF_XDP protocol family410gitlab # [ 0.907122] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.908182] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.909246] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.910420] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.911597] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.912983] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.914005] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.916425] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.919215] PCI: CLS 0 bytes, default 64419gitlab # [ 0.920068] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.920515] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.921168] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.921291] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 0.944559] Initialise system trusted keyrings424gitlab # [ 0.945729] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.956998] Key type asymmetric registered426gitlab # [ 0.957729] Asymmetric key parser 'x509' registered427gitlab # [ 0.958642] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.960327] io scheduler mq-deadline registered429gitlab # [ 0.961151] io scheduler kyber registered430gitlab # [ 0.965421] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.966772] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.969150] Linux agpgart interface v0.103433gitlab # [ 0.969960] ACPI: bus type drm_connector registered434gitlab # [ 0.972602] usbcore: registered new interface driver usbserial_generic435gitlab # [ 0.973707] usbserial: USB Serial support registered for generic436gitlab # [ 0.974709] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 0.974709] Please enable it if your BIOS has the CPPC option.438gitlab # [ 0.977048] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 0.978504] drop_monitor: Initializing network drop monitor service440gitlab # [ 0.979731] NET: Registered PF_INET6 protocol family441gitlab # [ 0.981584] Segment Routing with IPv6442gitlab # [ 0.982301] In-situ OAM (IOAM) with IPv6443gitlab # [ 0.984450] IPI shorthand broadcast: enabled444gitlab # [ 0.989019] sched_clock: Marking stable (843013865, 145884421)->(1014663927, -25765641)445gitlab # [ 0.990867] registered taskstats version 1446gitlab # [ 0.991915] Loading compiled-in X.509 certificates447gitlab # [ 1.001583] Demotion targets for Node 0: null448gitlab # [ 1.003278] Key type .fscrypt registered449gitlab # [ 1.004058] Key type fscrypt-provisioning registered450gitlab # [ 1.005064] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.006030] ima: Allocated hash algorithm: sha1452gitlab # [ 1.006863] ima: No architecture policies found453gitlab # [ 1.007880] PM: Magic number: 10:862:362454gitlab # [ 1.009607] RAS: Correctable Errors collector initialized.455gitlab # [ 1.014121] clk: Disabling unused clocks456gitlab # [ 1.014853] PM: genpd: Disabling unused power domains457gitlab # [ 1.222602] Freeing initrd memory: 29392K458gitlab # [ 1.229853] Freeing unused decrypted memory: 2028K459gitlab # [ 1.236312] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.238996] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.243831] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.247667] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.315824] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.317026] Run /init as init process465gitlab # [ 1.330301] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.353742] fuse: init (API version 7.45)467gitlab # [ 1.362186] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.369363] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.373114] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.410634] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.746615] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.763087] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.763977] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.764869] systemd[1]: Running in initrd.475gitlab # [ 1.766005] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.767244] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.888511] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.949007] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.965700] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.966935] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.968377] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.969555] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.970706] systemd[1]: Reached target Path Units.484gitlab # [ 1.971614] systemd[1]: Reached target Slice Units.485gitlab # [ 1.972515] systemd[1]: Reached target Swaps.486gitlab # [ 1.973396] systemd[1]: Reached target Timer Units.487gitlab # [ 1.974429] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.975686] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.976849] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.977852] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.979002] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.980011] systemd[1]: Reached target Socket Units.493gitlab # [ 1.982229] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.984331] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 1.986517] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 1.989432] systemd[1]: Starting Journal Service...497gitlab # [ 1.992972] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 1.995987] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 2.002106] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 2.019868] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.501gitlab # [ 2.021606] systemd[1]: Finished Create List of Static Device Nodes.502gitlab # [ 2.023013] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev503gitlab # [ 2.030498] systemd[1]: modprobe@configfs.service: Deactivated successfully.504gitlab # [ 2.032131] systemd[1]: Finished Load Kernel Module configfs.505gitlab # [ 2.033615] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config506gitlab # [ 2.036802] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...507gitlab # [ 2.050065] systemd-journald[92]: Collecting audit messages is disabled.508gitlab # [ 2.051377] systemd[1]: Finished Load Kernel Modules.509gitlab # [ 2.054572] systemd[1]: Starting Apply Kernel Variables...510gitlab # [ 2.057398] netfs: FS-Cache loaded511gitlab # [ 2.058066] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.512gitlab # [ 2.068336] 9pnet: Installing 9P2000 support513gitlab # [ 2.068940] systemd[1]: Starting Create Static Device Nodes in /dev...514gitlab # [ 2.072504] systemd[1]: Finished Apply Kernel Variables.515gitlab # [ 2.075426] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.516gitlab # [ 2.077409] systemd[1]: Finished Load Kernel Module 9pnet_virtio.517gitlab # [ 2.082273] systemd[1]: Finished Create Static Device Nodes in /dev.518gitlab # [ 2.083665] systemd[1]: Reached target Preparation for Local File Systems.519gitlab # [ 2.084955] systemd[1]: Reached target Local File Systems.520gitlab # [ 2.087085] systemd[1]: Starting Rule-based Manager for Device Events and Files...521gitlab # [ 2.090091] systemd[1]: Started Journal Service.522gitlab # [ 1.944302] systemd-modules-load[93]: Using 4 probe threads523gitlab # [ 1.946415] systemd-modules-load[93]: Inserted module 'virtio_balloon'524gitlab # [ 1.947532] systemd-modules-load[93]: Inserted module 'virtio_gpu'525gitlab # [ 1.948590] systemd-modules-load[93]: Inserted module 'dm_mod'526gitlab # [ 1.955063] systemd[1]: Starting Create System Files and Directories...527gitlab # [ 1.965113] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 1.968640] systemd-udevd[104]: Using default interface naming scheme 'v261'.529gitlab # [ 1.979040] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 1.999175] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.000166] systemd[1]: Reached target System Initialization.532gitlab # [ 2.000946] systemd[1]: Reached target Basic System.533gitlab # [ 2.228665] virtio_blk virtio4: 4/0/0 default/read/poll queues534gitlab # [ 2.234905] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)535gitlab # [ 2.234959] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12536gitlab # [ 2.237940] virtio_blk virtio5: 4/0/0 default/read/poll queues537gitlab # [ 2.242490] virtio_blk virtio5: [vdb] 11843216 512-byte logical blocks (6.06 GB/5.65 GiB)538gitlab # [ 2.246352] serio: i8042 KBD port at 0x60,0x64 irq 1539gitlab # [ 2.247558] serio: i8042 AUX port at 0x60,0x64 irq 12540gitlab # [ 2.254202] SCSI subsystem initialized541gitlab # [ 2.270448] uhci_hcd 0000:00:01.2: UHCI Host Controller542gitlab # [ 2.271267] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1543gitlab # [ 2.274225] uhci_hcd 0000:00:01.2: detected 2 ports544gitlab # [ 2.275611] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140545gitlab # [ 2.276792] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18546gitlab # [ 2.278084] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1547gitlab # [ 2.133672] (udev-worker)[138]: Network interface NamePolicy= disabled on kernel command line.548gitlab # [ 2.282923] usb usb1: Product: UHCI Host Controller549gitlab # [ 2.283746] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd550gitlab # [ 2.139057] systemd[1]: Starting Virtual Console Setup...551gitlab # [ 2.285589] usb usb1: SerialNumber: 0000:00:01.2552gitlab # [ 2.143505] (udev-worker)[124]: eth1: Config file /et[ 2.289916] hub 1-0:1.0: USB hub found553gitlab # c/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.554gitlab # [ 2.291558] hub 1-0:1.0: 2 ports detected555gitlab # [ 2.146832] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line.556gitlab # [ 2.303030] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0557gitlab # [ 2.308955] scsi host0: ata_piix558gitlab # [ 2.164452] systemd[1]: Found device /dev/disk/by-label/nixos.559gitlab # [ 2.310970] scsi host1: ata_piix560gitlab # [ 2.165633] systemd-vconsole-setup[143]: Configurati[ 2.311648] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0561gitlab # on of first virtual console was skipped, ignorin[ 2.313061] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0562gitlab # g remaining ones.563gitlab # [ 2.169283] systemd[1]: Found device /dev/disk/by-label/nix-store.564gitlab # [ 2.170276] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 2.171146] systemd[1]: Reached target Initrd Root Device.566gitlab # [ 2.171976] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...567gitlab # [ 2.197922] systemd-fsck[159]: nixos: clean, 12/65536 files, 13019/262144 blocks568gitlab # [ 2.201561] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.569gitlab # [ 2.475333] ata2: found unknown device (class 0)570gitlab # [ 2.478431] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100571gitlab # [ 2.482750] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5572gitlab # [ 2.514973] usb 1-1: new full-speed USB device number 2 using uhci_hcd573gitlab # [ 2.537362] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray574gitlab # [ 2.565615] cdrom: Uniform CD-ROM driver Revision: 3.20575gitlab # [ 2.682072] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.685230] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.687942] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.689554] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.691123] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.723157] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.737641] usbcore: registered new interface driver usbhid582gitlab # [ 2.739799] usbhid: USB HID core driver583gitlab # [ 2.749673] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.753848] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.873682] systemd[1]: Mounting /sysroot...586gitlab # [ 3.180969] EXT4-fs (vda): mounted filesystem c5e01e52-77ce-47c2-a1c3-6717f9119540 r/w with ordered data mode. Quota mode: none.587gitlab # [ 3.040166] systemd[1]: Mounted /sysroot.588gitlab # [ 3.042293] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 3.047518] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 3.051304] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.057936] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.060421] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.067385] systemd[1]: Mount[ 3.213815] erofs (device vdb): mounted with root inode @ nid 128.594gitlab # ing /sysroot/tmp/xchg...595gitlab # [ 3.070204] systemd[1]: Starting Mountpoints Configured in the Real Root...596gitlab # [ 3.075787] systemd[1]: Mounted /sysroot/nix/.ro-store.597gitlab # [ 3.078090] systemd-sysroot-fstab-check[180]: /sysroot should be mounted in the initrd, will request daemon-reload.598gitlab # [ 3.081127] systemd[1]: Mounted /sysroot/nix/.rw-store.599gitlab # [ 3.084578] systemd[1]: Mounted /sysroot/run.600gitlab # [ 3.087877] systemd[1]: Starting rw-sysroot-nix-store.service...601gitlab # [ 3.089435] systemd[1]: Reload requested from client PID 180 ('systemd-sysroot') (unit initrd-parse-etc.service)...602gitlab # [ 3.092257] systemd[1]: Reloading...603gitlab # [ 3.241680] 9p: Installing v9fs 9p2000 file system support604gitlab # [ 3.152669] systemd[1]: Reloading finished in 63 ms.605gitlab # [ 3.179779] systemd-sysroot-fstab-check[180]: Requesting initrd-fs.target/start/replace...606gitlab # [ 3.182611] systemd[1]: Mounted /sysroot/tmp/shared.607gitlab # [ 3.184557] systemd[1]: Mounted /sysroot/tmp/xchg.608gitlab # [ 3.186416] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.188996] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.191073] systemd-sysroot-fstab-check[180]: Requesting swap.target/start/replace...611gitlab # [ 3.193540] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.195603] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.198081] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.200634] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.203296] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.205652] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.874264] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.931837] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.934406] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.936404] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.938624] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.973615] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.976644] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.991742] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.993574] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.995620] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 4.025680] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 4.028055] systemd[1]: Stopped target Basic System.629gitlab # [ 4.029820] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 4.031766] systemd[1]: Stopped target Path Units.631gitlab # [ 4.033612] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 4.036462] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 4.041293] systemd[1]: Stopped target Slice Units.634gitlab # [ 4.044372] systemd[1]: Stopped target Socket Units.635gitlab # [ 4.046553] systemd[1]: Stopped target System Initialization.636gitlab # [ 4.049428] systemd[1]: Stopped target Swaps.637gitlab # [ 4.051151] systemd[1]: Stopped target Timer Units.638gitlab # [ 4.053269] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 4.055379] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 4.057416] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 4.059960] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.061903] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.065659] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.067527] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.069461] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.071058] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.072996] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.074584] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.076746] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.078929] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.081175] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.083083] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.084771] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.086717] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.087994] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.088934] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.090232] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.091433] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.092347] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.093479] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.094530] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.095695] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.096635] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.097654] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.098794] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.099959] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.100782] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.101614] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.102696] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.103624] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.104901] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.105921] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.107022] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.120933] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.122244] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.123555] systemd[1]: Reached target Switch Root.677gitlab # [ 4.125764] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.238639] initrd-nixos-activation-start[229]: booting system configuration /nix/store/6j8s7qi3n0s19hl82zlq8xpd9vb568i4-nixos-system-gitlab-test679gitlab # [ 4.276891] initrd-nixos-activation-start[229]: running activation script...680gitlab # [ 4.527330] initrd-nixos-activation-start[252]: setting up /etc...681gitlab # [ 4.633896] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.636582] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.638295] systemd[1]: Starting Switch Root...684gitlab # [ 4.670214] systemd[1]: Switching root.685gitlab # [ 4.882154] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.991975] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.361744] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.372012] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.373838] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.375858] systemd[1]: Detected first boot.691gitlab # [ 5.378488] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.526687] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.605135] systemd[1]: Applying preset policy.694gitlab # [ 5.741428] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.864392] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.865762] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.868173] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.870271] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.871661] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.873088] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.874045] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.875381] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.876546] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.877303] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.878092] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.879025] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.879787] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.880628] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.881486] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.882427] systemd[1]: Reached target Path Units.711gitlab # [ 5.883211] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.884051] systemd[1]: Reached target Slice Units.713gitlab # [ 5.884757] systemd[1]: Reached target Swaps.714gitlab # [ 5.886225] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.888254] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.889622] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.891279] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.892240] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.894578] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.896078] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.898003] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.899182] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.900366] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.901757] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.904558] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.905638] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.906575] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.909343] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.911356] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.913484] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.916333] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.926090] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.928710] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.938275] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.969208] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.972806] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.976021] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.980991] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.982739] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 5.989991] systemd[1]: Starting Journal Service...741gitlab # [ 5.991848] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 5.994195] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 5.998959] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 5.999994] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.004973] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.008197] systemd[1]: Listening on Journal Log Access Socket.747gitlab # [ 6.009252] systemd[1]: Mounted Huge Pages File System.748gitlab # [ 6.010314] systemd[1]: Mounted POSIX Message Queue File System.749gitlab # [ 6.011610] systemd[1]: Mounted Kernel Debug File System.750gitlab # [ 6.012720] systemd[1]: Mounted Kernel Trace File System.751gitlab # [ 6.014125] systemd[1]: Finished Create List of Static Device Nodes.752gitlab # [ 6.015389] systemd[1]: modprobe@configfs.service: Deactivated successfully.753gitlab # [ 6.016946] systemd[1]: Finished Load Kernel Module configfs.754gitlab # [ 6.018017] systemd[1]: Mounted FUSE Control File System.755gitlab # [ 6.019216] systemd-journald[322]: Collecting audit messages is enabled.756gitlab # [ 6.025005] systemd[1]: Mounting Kernel Configuration File System...757gitlab # [ 6.029190] EXT4-fs (vda): re-mounted c5e01e52-77ce-47c2-a1c3-6717f9119540.758gitlab # [ 5.890605] systemd[1]: Queued start job for default target Multi-User System.759gitlab # [ 6.039257] loop: module loaded760gitlab # [ 5.894961] systemd[1]: Unnecessary job was removed for /dev/vdb.761gitlab # [ 6.040734] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...762gitlab # [ 5.896288] systemd[1]: systemd-journald.service: Deactivated successfully.763gitlab # [ 5.898934] systemd-modules-load[323]: Using 3 probe threads764gitlab # [ 6.045991] systemd[1]: Started Journal Service.765gitlab # [ 5.902189] systemd-modules-load[323]: Inserted module 'tls'766gitlab # [ 5.903232] systemd-modules-load[323]: Inserted module 'loop'767gitlab # [ 5.904431] systemd-oomd[324]: No swap; memory pressure usage will be degraded768gitlab # [ 5.905896] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.769gitlab # [ 5.912102] systemd[1]: Finished Load Kernel Modules.770gitlab # [ 5.915860] systemd[1]: Finished Remount Root and Kernel File Systems.771gitlab # [ 5.917800] systemd[1]: Mounted Kernel Configuration File System.772gitlab # [ 5.924774] systemd[1]: Listening on Disk Image Download Service Socket.773gitlab # [ 5.925914] systemd[1]: Starting Firewall...774gitlab # [ 5.927131] systemd[1]: Starting Flush Journal to Persistent Storage...775gitlab # [ 5.928725] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore776gitlab # [ 5.930300] systemd[1]: Starting Load/Save OS Random Seed...777gitlab # [ 5.938909] systemd[1]: Starting Apply Kernel Variables...778gitlab # [ 5.939856] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os779gitlab # [ 5.942217] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.780gitlab # [ 5.944426] systemd[1]: Starting Create Static Device Nodes in /dev...781gitlab # [ 6.092836] systemd-journald[322]: Received client request to flush runtime journal.782gitlab # [ 5.989449] systemd[1]: Finished Load/Save OS Random Seed.783gitlab # [ 5.991308] systemd[1]: Finished Apply Kernel Variables.784gitlab # [ 5.992256] systemd[1]: Reached target First Boot Complete.785gitlab # [ 5.993254] systemd[1]: Finished Create Static Device Nodes in /dev.786gitlab # [ 5.994338] systemd[1]: Reached target Preparation for Local File Systems.787gitlab # [ 5.995493] systemd[1]: Starting Rule-based Manager for Device Events and Files...788gitlab # [ 5.996546] systemd[1]: Finished Flush Journal to Persistent Storage.789gitlab # [ 6.002968] systemd-udevd[357]: Using default interface naming scheme 'v261'.790gitlab # [ 6.025429] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.036216] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.099099] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.133082] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.134437] (udev-worker)[436]: Network interface NamePolicy= disabled on kernel command line.795gitlab # [ 6.142371] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.796gitlab # [ 6.147507] (udev-worker)[435]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 6.149784] (udev-worker)[435]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.160112] systemd[1]: Condition check resulted in Virtio network device being skipped.799gitlab # [ 6.161816] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore800gitlab # [ 6.163813] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67801gitlab # [ 6.166273] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore802gitlab # [ 6.168196] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os803gitlab # [ 6.169733] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.340328] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3805gitlab # [ 6.349824] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console806gitlab # [ 6.356488] mousedev: PS/2 mouse device common for all mice807gitlab # [ 6.362108] Console: switching to colour dummy device 80x25808gitlab # [ 6.364372] [drm] Found bochs VGA, ID 0xb0c5.809gitlab # [ 6.364373] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.810gitlab # [ 6.366417] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic811gitlab # [ 6.367234] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0812gitlab # [ 6.369004] ACPI: button: Power Button [PWRF]813gitlab # [ 6.369859] Floppy drive(s): fd0 is 2.88M AMI BIOS814gitlab # [ 6.373070] rtc_cmos 00:05: RTC can wake from S4815gitlab # [ 6.374635] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4816gitlab # [ 6.377137] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0817gitlab # [ 6.379583] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD818gitlab # [ 6.383912] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6819gitlab # [ 6.385543] rtc_cmos 00:05: registered as rtc0820gitlab # [ 6.385586] FDC 0 is a S82078B821gitlab # [ 6.385625] rtc_cmos 00:05: setting system clock to 2026-09-08T04:23:01 UTC (1788841381)822gitlab # [ 6.385665] systemd-journald[322]: Time jumped backwards, rotating.823gitlab # [ 6.385734] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs824gitlab # [ 6.386993] parport_pc 00:03: reported by Plug and Play ACPI825gitlab # [ 6.387099] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]826gitlab # [ 6.387228] Console: switching to colour frame buffer device 160x50827gitlab # [ 6.395841] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device828gitlab # [ 6.398670] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5829gitlab # [ 6.260789] systemd[1]: Finished Firewall.830gitlab # [ 6.322546] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.342921] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.832gitlab # [ 6.345990] systemd[1]: Stopped Virtual Console Setup.833gitlab # [ 6.349165] systemd[1]: Starting Virtual Console Setup...834gitlab # [ 6.368247] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.835gitlab # [ 6.369367] systemd[1]: Stopped Virtual Console Setup.836gitlab # [ 6.372882] systemd[1]: Starting Virtual Console Setup...837gitlab # [ 6.524867] ppdev: user-space parallel port driver838gitlab # [ 6.533446] kvm_amd: TSC scaling supported839gitlab # [ 6.534021] kvm_amd: Nested Virtualization enabled840gitlab # [ 6.534537] kvm_amd: Nested Paging enabled841gitlab # [ 6.535243] kvm_amd: LBR virtualization supported842gitlab # [ 6.535849] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 6.536567] kvm_amd: Virtual GIF supported844gitlab # [ 6.537152] kvm_amd: Virtual NMI enabled845gitlab # [ 6.568433] EDAC MC: Ver: 3.0.0846gitlab # [ 6.447059] systemd-vconsole-setup[500]: Configuration of first virtual console was skipped, ignoring remaining ones.847gitlab # [ 6.448717] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 6.724277] systemd[1]: Mounting /run/wrappers...849gitlab # [ 6.753592] systemd[1]: Mounted /run/wrappers.850gitlab # [ 6.755496] systemd[1]: Reached target Local File Systems.851gitlab # [ 6.757200] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 6.759564] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 6.761783] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 6.763565] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 6.766342] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 6.772209] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 6.796423] systemd[1]: Finished Create System Files and Directories.858gitlab # [ 6.801624] systemd[1]: Starting Rebuild Journal Catalog...859gitlab # [ 6.804619] systemd[1]: Starting Record System Boot/Shutdown in UTMP...860gitlab # [ 6.822214] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.861gitlab # [ 6.824318] systemd[1]: Finished Save Transient machine-id to Disk.862gitlab # [ 6.830525] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 6.843537] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 6.845200] systemd[1]: Starting Update is Completed...865gitlab # [ 6.861600] systemd[1]: Finished Update is Completed.866gitlab # [ 6.934964] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 6.936184] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 7.173212] systemd[1]: Finished register-nix-paths.service.869gitlab # [ 7.174523] systemd[1]: Reached target System Initialization.870gitlab # [ 7.175655] systemd[1]: Started Discard unused filesystem blocks once a week.871gitlab # [ 7.176906] systemd[1]: Started logrotate.timer.872gitlab # [ 7.177804] systemd[1]: Started Daily Cleanup of Temporary Directories.873gitlab # [ 7.178931] systemd[1]: Reached target Timer Units.874gitlab # [ 7.179873] systemd[1]: Listening on D-Bus System Message Bus Socket.875gitlab # [ 7.180960] systemd[1]: Listening on Nix Daemon Socket.876gitlab # [ 7.182265] systemd[1]: Starting nixbot CI service socket...877gitlab # [ 7.183332] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.878gitlab # [ 7.184751] systemd[1]: Listening on nixbot CI service socket.879gitlab # [ 7.186056] systemd[1]: Reached target Socket Units.880gitlab # [ 7.188403] systemd[1]: Starting D-Bus System Message Bus...881gitlab # [ 7.244158] dbus-broker-launch[584]: Looking up NSS user entry for 'systemd-timesync'...882gitlab # [ 7.246678] dbus-broker-launch[584]: NSS returned no entry for 'systemd-timesync'883gitlab # [ 7.249420] dbus-broker-launch[584]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"884gitlab # [ 7.255278] systemd[1]: Started D-Bus System Message Bus.885gitlab # [ 7.269154] dbus-broker-launch[584]: Ready886gitlab # [ 7.273302] systemd[1]: Reached target Basic System.887gitlab # [ 7.276205] systemd[1]: Started backdoor.service.888gitlab # [ 7.278822] systemd[1]: Starting gitlab-config.service...889gitlab # [ 7.281410] systemd[1]: Starting Import lastlog data into lastlog2 database...890gitlab # [ 7.290153] systemd[1]: Starting Logrotate configuration check...891gitlab # [ 7.303186] systemd[1]: Starting Name Service Cache Daemon (nsncd)...892gitlab # [ 7.308132] systemd[1]: Starting Post-Boot Actions...893gitlab # [ 7.311448] systemd[1]: Started Reset console on configuration changes.894gitlab # [ 7.316032] systemd[1]: Starting resolvconf update...895gitlab # connecting to host...896gitlab # [ 7.321245] logrotate[589]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.897gitlab # [ 7.321385] logrotate[589]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf898gitlab # [ 7.322066] logrotate[589]: note: 'monthly' overrides previously specified 'weekly'899gitlab # [ 7.322359] logrotate[589]: Reading state from file: /var/lib/logrotate.status900gitlab # [ 7.322896] logrotate[589]: state file /var/lib/logrotate.status does not exist901gitlab # [ 7.323383] logrotate[589]: Allocating hash table for state file, size 64 entries902gitlab # [ 7.323873] logrotate[589]: Handling 4 logs903gitlab # [ 7.326873] logrotate[589]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed904gitlab # [ 7.327487] logrotate[589]: considering log /var/log/btmp905gitlab # [ 7.328025] logrotate[589]: Creating new state906gitlab # [ 7.328448] logrotate[589]: Now: 2026-09-08 04:23907gitlab # [ 7.328936] logrotate[589]: Last rotated at 2026-09-08 04:00908gitlab # [ 7.329594] logrotate[589]: log does not need rotating (log has already been rotated)909gitlab # [ 7.330426] logrotate[589]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed910gitlab # [ 7.330916] logrotate[589]: considering log /var/log/wtmp911gitlab # [ 7.331563] logrotate[589]: Creating new state912gitlab # [ 7.334781] logrotate[589]: Now: 2026-09-08 04:23913gitlab # [ 7.334944] logrotate[589]: Last rotated at 2026-09-08 04:00914gitlab # [ 7.335558] logrotate[589]: log does not need rotating (log has already been rotated)915gitlab # [ 7.336075] logrotate[589]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed916gitlab # [ 7.336503] logrotate[589]: switching euid from 0 to 165 and egid from 0 to 165 (pid 589)917gitlab # [ 7.337024] logrotate[589]: considering log /var/gitlab/state/log/*.log918gitlab # [ 7.337487] logrotate[589]: log /var/gitlab/state/log/*.log does not exist -- skipping919gitlab # [ 7.337993] logrotate[589]: Creating new state920gitlab # [ 7.338434] logrotate[589]: switching euid from 165 to 0 and egid from 165 to 0 (pid 589)921gitlab # [ 7.338896] logrotate[589]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed922gitlab # [ 7.339563] logrotate[589]: switching euid from 0 to 60 and egid from 0 to 60 (pid 589)923gitlab # [ 7.340148] logrotate[589]: considering log /var/log/nginx/*.log924gitlab # [ 7.340607] logrotate[589]: log /var/log/nginx/*.log does not exist -- skipping925gitlab # [ 7.341134] logrotate[589]: Creating new state926gitlab # [ 7.341538] logrotate[589]: not running postrotate script, since no logs were rotated927gitlab # [ 7.342075] logrotate[589]: switching euid from 60 to 0 and egid from 60 to 0 (pid 589)928gitlab # [ 7.346266] systemd[1]: Finished Logrotate configuration check.929gitlab # [ 7.347192] systemd[1]: Finished Post-Boot Actions.930gitlab # [ 7.358146] systemd[1]: Finished Import lastlog data into lastlog2 database.931gitlab: Guest shell says: b'Spawning backdoor root shell...\n'932gitlab # [ 7.372288] systemd[1]: Started Name Service Cache Daemon (nsncd).933gitlab: connected to guest root shell934gitlab # [ 7.373254] systemd[1]: Reached target Host and Network Name Lookups.935gitlab: (connecting took 8.04 seconds)936gitlab # [ 7.374159] systemd[1]: Reached target User and Group Name Lookups.937gitlab: (finished: waiting for the VM to finish booting, in 8.04 seconds)938gitlab # [ 7.375855] nsncd[590]: Sep 08 04:23:02.632 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"939gitlab # [ 7.378352] systemd[1]: Starting User Login Management...940gitlab # [ 7.398860] systemd[1]: Stopped target Host and Network Name Lookups.941gitlab # [ 7.399514] systemd[1]: Stopping Host and Network Name Lookups...942gitlab # [ 7.400074] systemd[1]: Stopped target User and Group Name Lookups.943gitlab # [ 7.400604] systemd[1]: Stopping User and Group Name Lookups...944gitlab # [ 7.402100] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...945gitlab # [ 7.405143] systemd[1]: nscd.service: Deactivated successfully.946gitlab # [ 7.405279] systemd[1]: Stopped Name Service Cache Daemon (nsncd).947gitlab # [ 7.405836] systemd[1]: nscd.service: Consumed 22ms CPU time over 101ms wall clock time, 7.4M memory peak, 5.5M read from disk.948gitlab # [ 7.411253] systemd[1]: Starting Name Service Cache Daemon (nsncd)...949gitlab # [ 7.423132] systemd-logind[647]: New seat seat0.950gitlab # [ 7.425654] systemd-logind[647]: Watching system buttons on /dev/input/event2 (Power Button)951gitlab # [ 7.426834] systemd-logind[647]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)952gitlab # [ 7.427271] systemd-logind[647]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)953gitlab # [ 7.428168] systemd[1]: Started User Login Management.954gitlab # [ 7.433344] nsncd[681]: Sep 08 04:23:02.690 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"955gitlab # [ 7.437821] systemd[1]: Starting linger-users.service...956gitlab # [ 7.439517] systemd[1]: Started Name Service Cache Daemon (nsncd).957gitlab # [ 7.440659] systemd[1]: Reached target Host and Network Name Lookups.958gitlab # [ 7.441684] systemd[1]: Reached target User and Group Name Lookups.959gitlab # [ 7.442662] systemd[1]: Finished resolvconf update.960gitlab # [ 7.443474] systemd[1]: Reached target Preparation for Network.961gitlab # [ 7.444456] systemd[1]: Starting DHCP Client...962gitlab # [ 7.445575] systemd[1]: Starting Address configuration of eth1...963gitlab # [ 7.447138] systemd[1]: Starting Extra networking commands....964gitlab # [ 7.452099] systemd[1]: linger-users.service: Deactivated successfully.965gitlab # [ 7.458138] systemd[1]: Finished linger-users.service.966gitlab # [ 7.476099] network-addresses-eth1-start[706]: adding address 192.168.1.1/24... done967gitlab # [ 7.479305] network-addresses-eth1-start[706]: adding address 2001:db8:1::1/64... done968gitlab # [ 7.483613] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 7.508799] systemd[1]: Finished Extra networking commands..970gitlab # [ 7.510271] systemd[1]: Reached target Network.971gitlab # [ 7.511626] systemd[1]: Starting gitlab-workhorse.service...972gitlab # [ 7.513441] systemd[1]: Starting Nginx Web Server...973gitlab # [ 7.515599] systemd[1]: Starting PostgreSQL Server...974gitlab # [ 7.520718] systemd[1]: Starting Redis Server - redis-gitlab...975gitlab # [ 7.523168] systemd[1]: Starting Permit User Sessions...976gitlab # [ 7.525216] dhcpcd[716]: dhcpcd-10.3.2 starting977gitlab # [ 7.527653] dhcpcd[778]: dev: loaded udev978gitlab # [ 7.682474] 8021q: 802.1Q VLAN Support v1.8979gitlab # [ 7.684713] 8021q: adding VLAN 0 to HW filter on device eth1980gitlab # [ 7.538920] systemd[1]: Finished Permit User Sessions.981gitlab # [ 7.543391] systemd[1]: Started Getty on tty1.982gitlab # [ 7.545719] systemd[1]: Reached target Login Prompts.983gitlab # [ 7.553205] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.984gitlab # [ 7.764324] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 7.772126] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 7.773148] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 7.777160] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 7.778694] cfg80211: failed to load regulatory.db989gitlab # [ 7.647387] nginx-pre-start[799]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok990gitlab # [ 7.647611] nginx-pre-start[799]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful991gitlab # [ 7.654195] systemd[1]: Started Nginx Web Server.992gitlab # [ 7.807893] 8021q: adding VLAN 0 to HW filter on device eth0993gitlab # [ 7.664661] dhcpcd[778]: eth0: waiting for carrier994gitlab # [ 7.665110] dhcpcd[778]: eth0: carrier acquired995gitlab # [ 7.667972] redis-server[797]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.996gitlab # [ 7.668402] redis-server[797]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo997gitlab # [ 7.668993] redis-server[797]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=797, just started998gitlab # [ 7.669495] redis-server[797]: Configuration loaded999gitlab # [ 7.669987] redis[797]: monotonic clock: POSIX clock_gettime1000gitlab # [ 7.671165] redis[797]: Running mode=standalone, port=0.1001gitlab # [ 7.671740] redis[797]: Server initialized1002gitlab # [ 7.672469] systemd[1]: Started Redis Server - redis-gitlab.1003gitlab # [ 7.673097] redis[797]: Ready to accept connections unix1004gitlab # [ 7.673675] redis[797]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1005gitlab # [ 7.674273] dhcpcd[778]: DUID 00:01:00:01:32:32:4a:26:52:54:00:12:34:561006gitlab # [ 7.674832] dhcpcd[778]: eth0: IAID 00:12:34:561007gitlab # [ 7.675531] dhcpcd[778]: eth0: adding address fe80::5054:ff:fe12:34561008gitlab # [ 7.685234] postgresql-pre-start[800]: The files belonging to this database system will be owned by user "postgres".1009gitlab # [ 7.685518] postgresql-pre-start[800]: This user must also own the server process.1010gitlab # [ 7.690609] postgresql-pre-start[800]: The database cluster will be initialized with locale "en_US.UTF-8".1011gitlab # [ 7.690958] postgresql-pre-start[800]: The default database encoding has accordingly been set to "UTF8".1012gitlab # [ 7.691584] postgresql-pre-start[800]: The default text search configuration will be set to "english".1013gitlab # [ 7.692147] postgresql-pre-start[800]: Data page checksums are enabled.1014gitlab # [ 7.692678] postgresql-pre-start[800]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1015gitlab # [ 7.696600] postgresql-pre-start[800]: creating subdirectories ... ok1016gitlab # [ 7.696869] postgresql-pre-start[800]: selecting dynamic shared memory implementation ... posix1017gitlab # [ 7.732393] postgresql-pre-start[800]: selecting default "max_connections" ... 1001018gitlab # [ 7.767964] postgresql-pre-start[800]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 8.026831] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 8.185153] postgresql-pre-start[800]: selecting default time zone ... UTC1021gitlab # [ 8.187111] postgresql-pre-start[800]: creating configuration files ... ok1022gitlab # [ 8.226752] gitlab-workhorse[833]: time="2026-09-08T04:23:03Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41023gitlab # [ 8.227171] gitlab-workhorse[833]: time="2026-09-08T04:23:03Z" level=info msg="Using redis/go-redis"1024gitlab # [ 8.227619] gitlab-workhorse[833]: time="2026-09-08T04:23:03Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 8.352286] postgresql-pre-start[800]: running bootstrap script ... ok1026gitlab # [ 8.791866] postgresql-pre-start[800]: performing post-bootstrap initialization ... ok1027gitlab # [ 9.070093] dhcpcd[778]: eth0: soliciting a DHCP lease1028gitlab # [ 9.221304] NET: Registered PF_PACKET protocol family1029gitlab # [ 9.084883] dhcpcd[778]: eth0: offered 10.0.2.15 from 10.0.2.21030gitlab # [ 9.089627] dhcpcd[778]: eth0: probing address 10.0.2.15/241031gitlab # [ 9.096484] postgresql-pre-start[800]: syncing data to disk ... ok1032gitlab # [ 9.097169] postgresql-pre-start[800]: initdb: warning: enabling "trust" authentication for local connections1033gitlab # [ 9.098255] postgresql-pre-start[800]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1034gitlab # [ 9.099381] postgresql-pre-start[800]: Success. You can now start the database server using:1035gitlab # [ 9.100819] postgresql-pre-start[800]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1036gitlab # [ 9.196563] postgres[861]: [861] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1037gitlab # [ 9.200658] postgres[861]: [861] LOG: listening on IPv6 address "::1", port 54321038gitlab # [ 9.201402] postgres[861]: [861] LOG: listening on IPv4 address "127.0.0.1", port 54321039gitlab # [ 9.210387] postgres[861]: [861] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1040gitlab # [ 9.228277] postgres[870]: [870] LOG: database system was shut down at 2026-09-08 04:23:04 GMT1041gitlab # [ 9.234971] postgres[861]: [861] LOG: database system is ready to accept connections1042gitlab # [ 9.242781] systemd[1]: Started PostgreSQL Server.1043gitlab # [ 9.246885] systemd[1]: Starting PostgreSQL Setup Scripts...1044gitlab # [ 9.315536] dhcpcd[778]: eth0: soliciting an IPv6 router1045gitlab # [ 9.318248] dhcpcd[778]: eth0: Router Advertisement from fe80::21046gitlab # [ 9.320269] dhcpcd[778]: eth0: adding address fec0::5054:ff:fe12:3456/641047gitlab # [ 9.322336] dhcpcd[778]: eth0: adding route to fec0::/641048gitlab # [ 9.322482] dhcpcd[778]: eth0: adding default route via fe80::21049gitlab # [ 9.464583] postgresql-setup-start[881]: CREATE DATABASE1050gitlab # [ 9.493438] postgresql-setup-start[899]: CREATE ROLE1051gitlab # [ 9.502233] postgresql-setup-start[902]: ALTER DATABASE1052gitlab # [ 9.525352] postgresql-setup-start[907]: CREATE ROLE1053gitlab # [ 9.528652] systemd[1]: Finished PostgreSQL Setup Scripts.1054gitlab # [ 9.529513] systemd[1]: Reached target PostgreSQL.1055gitlab # [ 9.531151] systemd[1]: Starting gitlab-postgresql.service...1056gitlab # [ 9.616733] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[890]: mkdir -p /var/gitlab/state/home/.ssh: OK1057gitlab # [ 9.622117] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[890]: chmod 700 /var/gitlab/state/home/.ssh: OK1058gitlab # [ 9.641951] gitlab-postgresql-start[915]: CREATE DATABASE1059gitlab # [ 9.661102] systemd[1]: Finished gitlab-config.service.1060gitlab # [ 9.663244] systemd[1]: Started gitaly.service.1061gitlab # [ 9.664928] systemd[1]: Starting gitlab-db-config.service...1062gitlab # [ 9.682584] gitlab-postgresql-start[931]: CREATE EXTENSION1063gitlab # [ 9.766275] gitlab-postgresql-start[936]: CREATE EXTENSION1064gitlab # [ 9.768253] systemd[1]: Finished gitlab-postgresql.service.1065gitlab # [ 10.163450] gitaly[932]: time="2026-09-08T04:23:05.422Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9321066gitlab # [ 10.164369] gitaly[932]: time="2026-09-08T04:23:05.423Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9321067gitlab # [ 10.165267] gitaly[932]: time="2026-09-08T04:23:05.423Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9321068gitlab # [ 10.165989] gitaly[932]: time="2026-09-08T04:23:05.423Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9321069gitlab # [ 10.166705] gitaly[932]: time="2026-09-08T04:23:05.423Z" level=info msg="Starting Gitaly" pid=932 version=1070gitlab # [ 10.328990] gitaly[932]: time="2026-09-08T04:23:05.589Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=165 pid=9321071gitlab # [ 10.329404] gitaly[932]: time="2026-09-08T04:23:05.589Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9321072gitlab # [ 10.330345] gitaly[932]: time="2026-09-08T04:23:05.590Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9321073gitlab # [ 10.330988] gitaly[932]: time="2026-09-08T04:23:05.590Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-932/git-exec-3651065561.d/git pid=9321074gitlab # [ 10.335581] gitaly[932]: time="2026-09-08T04:23:05.595Z" level=info msg="finished detecting git version" duration_ms=5 pid=9321075gitlab # [ 10.336172] gitaly[932]: time="2026-09-08T04:23:05.595Z" level=info msg="using Git version" pid=932 version=2.53.g1076gitlab # [ 10.339355] gitaly[932]: time="2026-09-08T04:23:05.596Z" level=info msg="clearing disk cache object folder" pid=932 storage=default1077gitlab # [ 10.343820] gitaly[932]: time="2026-09-08T04:23:05.596Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1544277586 pid=932 storage=default1078gitlab # [ 10.344140] gitaly[932]: time="2026-09-08T04:23:05.596Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1544277586 pid=932 storage=default1079gitlab # [ 10.344629] gitaly[932]: time="2026-09-08T04:23:05.596Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9321080gitlab # [ 10.345128] gitaly[932]: time="2026-09-08T04:23:05.596Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9321081gitlab # [ 10.345621] gitaly[932]: time="2026-09-08T04:23:05.597Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1544277586 clear_duration_ms=0 pid=932 storage=default1082gitlab # [ 10.387037] gitaly[932]: time="2026-09-08T04:23:05.647Z" level=info msg="finished tempdir cleaner walk" pid=932 storage=default time_ms=01083gitlab # [ 10.387417] gitaly[932]: time="2026-09-08T04:23:05.647Z" level=info msg="counting repositories" pid=9321084gitlab # [ 10.390439] gitaly[932]: time="2026-09-08T04:23:05.647Z" level=info msg="starting to count repositories" pid=932 storage_path=/var/gitlab/state/repositories1085gitlab # [ 10.390540] gitaly[932]: time="2026-09-08T04:23:05.647Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9321086gitlab # [ 10.390791] gitaly[932]: time="2026-09-08T04:23:05.650Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=932 storage_path=/var/gitlab/state/repositories1087gitlab # [ 10.390967] gitaly[932]: time="2026-09-08T04:23:05.650Z" level=info msg="completed counting all repositories" counting_duration=3 pid=9321088gitlab # [ 10.391840] gitaly[932]: time="2026-09-08T04:23:05.651Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=932 schema=unix1089gitlab # [ 10.392558] gitaly[932]: time="2026-09-08T04:23:05.651Z" level=info msg="listening at address" address=/run/gitaly/gitaly-932/sock.d/intern pid=932 schema=unix1090gitlab # [ 10.404229] gitaly[932]: time="2026-09-08T04:23:05.664Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9321091gitlab # [ 10.404724] gitaly[932]: time="2026-09-08T04:23:05.664Z" level=info msg="Started Gitaly" duration_ms=241 pid=9321092gitlab # [ 10.405402] gitaly[932]: time="2026-09-08T04:23:05.664Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9321093gitlab # [ 10.405825] gitaly[932]: time="2026-09-08T04:23:05.664Z" level=info msg="maintenance: daily scheduled" pid=932 scheduled="2026-09-08 12:00:00 +0000 UTC"1094gitlab # [ 11.432116] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[938]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1095gitlab # [ 11.438469] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[938]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1096gitlab # [ 11.461700] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[938]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1097gitlab # [ 12.335434] gitaly[932]: time="2026-09-08T04:23:07.595Z" level=info msg="License database preloaded" duration_ms=2171 pid=9321098gitlab # [ 14.115184] dhcpcd[778]: eth0: leased 10.0.2.15 for 86400 seconds1099gitlab # [ 14.115999] dhcpcd[778]: eth0: adding route to 10.0.2.0/241100gitlab # [ 14.117142] dhcpcd[778]: eth0: adding default route via 10.0.2.21101gitlab # [ 14.228171] systemd[1]: Started DHCP Client.1102gitlab # [ 14.229915] systemd[1]: Reached target Network is Online.1103gitlab # [ 43.041115] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[938]: W, [2026-09-08T04:23:38.300743 #938] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 45.285815] postgres[1334]: [1334] ERROR: relation "feature_gates" does not exist at character 5231105gitlab # [ 45.287293] postgres[1334]: [1334] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1106gitlab # [ 45.288498] postgres[1334]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1107gitlab # [ 45.291093] postgres[1334]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1108gitlab # [ 45.293533] postgres[1334]: attidentity AS identity,1109gitlab # [ 45.295178] postgres[1334]: attgenerated as attgenerated1110gitlab # [ 45.296821] postgres[1334]: FROM pg_attribute a1111gitlab # [ 45.298303] postgres[1334]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1112gitlab # [ 45.300742] postgres[1334]: LEFT JOIN pg_type t ON a.atttypid = t.oid1113gitlab # [ 45.302603] postgres[1334]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1114gitlab # [ 45.305315] postgres[1334]: WHERE a.attrelid = '"feature_gates"'::regclass1115gitlab # [ 45.307235] postgres[1334]: AND a.attnum > 0 AND NOT a.attisdropped1116gitlab # [ 45.308989] postgres[1334]: ORDER BY a.attnum1117gitlab # [ 45.309721] postgres[1334]:1118gitlab # [ 48.272236] gitlab-workhorse[833]: time="2026-09-08T04:23:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1119gitlab # [ 48.941794] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1376]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1120gitlab # [ 48.942063] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1376]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1121gitlab # [ 58.273672] gitlab-workhorse[833]: time="2026-09-08T04:23:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 63.674254] postgres[1442]: [1442] ERROR: relation "table_without_models" does not exist1123gitlab # [ 63.674776] postgres[1442]: [1442] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1124gitlab # [ 68.275333] gitlab-workhorse[833]: time="2026-09-08T04:24:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 75.927488] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 75.933486] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 75.938332] systemd[1]: Starting gitlab.service...1128gitlab # [ 76.291331] bundle[1618]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 76.291895] bundle[1618]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 76.293389] bundle[1618]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 78.285254] gitlab-workhorse[833]: time="2026-09-08T04:24:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1132gitlab # [ 79.038971] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-09-08T04:24:14.281Z","message":"Starting cluster with 1 processes"}1133gitlab # [ 79.166735] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.406Z","pid":1618,"message":"* Enabling systemd notification integration"}1134gitlab # [ 79.170558] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.425Z","pid":1618,"message":"Puma starting in cluster mode..."}1135gitlab # [ 79.173975] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.425Z","pid":1618,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1136gitlab # [ 79.177670] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1137gitlab # [ 79.182360] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Min threads: 0"}1138gitlab # [ 79.185473] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Max threads: 4"}1139gitlab # [ 79.188454] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Environment: production"}1140gitlab # [ 79.191705] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Master PID: 1618"}1141gitlab # [ 79.194758] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Workers: 2"}1142gitlab # [ 79.197718] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1143gitlab # [ 79.201474] bundle[1618]: {"timestamp":"2026-09-08T04:24:14.426Z","pid":1618,"message":"* Preloading application"}1144gitlab # [ 88.303186] gitlab-workhorse[833]: time="2026-09-08T04:24:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 98.313695] gitlab-workhorse[833]: time="2026-09-08T04:24:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 102.151466] bundle[1618]: W, [2026-09-08T04:24:37.410715 #1618] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1147gitlab # [ 104.466800] sidekiq-cluster[1648]: 2026-09-08T04:24:39.726Z pid=1648 tid=wk8 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1648, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>435445760}1148gitlab # [ 104.642085] sidekiq-cluster[1648]: W, [2026-09-08T04:24:39.902038 #1648] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1149gitlab # [ 104.680755] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:39.940Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1150gitlab # [ 106.641214] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.900Z","message":"GitLab reliable fetch activated!"}1151gitlab # [ 106.641487] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.900Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1152gitlab # [ 106.676798] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.936Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1153gitlab # [ 106.682450] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.938Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1154gitlab # [ 106.687849] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.940Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1155gitlab # [ 106.692897] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.942Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1156gitlab # [ 106.698578] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.944Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1157gitlab # [ 106.703819] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.946Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1158gitlab # [ 106.709089] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.948Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1159gitlab # [ 106.714051] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.950Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1160gitlab # [ 106.719141] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.951Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1161gitlab # [ 106.724656] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.953Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1162gitlab # [ 106.730195] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.955Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1163gitlab # [ 106.734621] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.957Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1164gitlab # [ 106.738429] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.958Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1165gitlab # [ 106.740700] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.960Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1166gitlab # [ 106.742861] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.961Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1167gitlab # [ 106.745143] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.963Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1168gitlab # [ 106.747210] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.965Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1169gitlab # [ 106.749243] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.967Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1170gitlab # [ 106.751402] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.969Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1171gitlab # [ 106.753539] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.971Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1172gitlab # [ 106.755826] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.973Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1173gitlab # [ 106.758269] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.975Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1174gitlab # [ 106.760342] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.976Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1175gitlab # [ 106.762590] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.978Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1176gitlab # [ 106.764832] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.980Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1177gitlab # [ 106.767103] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.981Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1178gitlab # [ 106.769174] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.983Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1179gitlab # [ 106.771348] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.985Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1180gitlab # [ 106.773477] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.987Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1181gitlab # [ 106.775629] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.989Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1182gitlab # [ 106.777632] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.991Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1183gitlab # [ 106.779854] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.993Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1184gitlab # [ 106.781895] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.995Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1185gitlab # [ 106.784046] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.998Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1186gitlab # [ 106.786298] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:41.999Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1187gitlab # [ 106.788473] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.001Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1188gitlab # [ 106.790627] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.003Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1189gitlab # [ 106.792733] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.005Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1190gitlab # [ 106.794861] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.007Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1191gitlab # [ 106.796885] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.008Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1192gitlab # [ 106.799152] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.010Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1193gitlab # [ 106.801436] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.012Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1194gitlab # [ 106.804312] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.014Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1195gitlab # [ 106.806846] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.015Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1196gitlab # [ 106.807294] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.017Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1197gitlab # [ 106.807721] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.019Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1198gitlab # [ 106.808477] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.021Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1199gitlab # [ 106.809884] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.023Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1200gitlab # [ 106.810933] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.025Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1201gitlab # [ 106.811581] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.026Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1202gitlab # [ 106.812661] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.028Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1203gitlab # [ 106.813876] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.030Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1204gitlab # [ 106.814781] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.039Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1205gitlab # [ 106.815502] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.041Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1206gitlab # [ 106.816276] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.043Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1207gitlab # [ 106.817091] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.045Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1208gitlab # [ 106.817539] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.047Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1209gitlab # [ 106.818218] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.050Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1210gitlab # [ 106.818667] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.051Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1211gitlab # [ 106.819549] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.053Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1212gitlab # [ 106.820233] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.054Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1213gitlab # [ 106.820943] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.056Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1214gitlab # [ 106.821597] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.058Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1215gitlab # [ 106.822148] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.060Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1216gitlab # [ 106.822718] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.062Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1217gitlab # [ 106.823685] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.064Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1218gitlab # [ 106.824584] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.069Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1219gitlab # [ 106.825156] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.074Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1220gitlab # [ 106.825819] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.077Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1221gitlab # [ 106.826548] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.079Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1222gitlab # [ 106.827291] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.081Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1223gitlab # [ 106.827794] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.084Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1224gitlab # [ 106.828526] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.087Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1225gitlab # [ 106.829541] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.089Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1226gitlab # [ 106.831971] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.092Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1227gitlab # [ 106.834476] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.094Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1228gitlab # [ 106.836819] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.097Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1229gitlab # [ 106.839108] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.099Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1230gitlab # [ 106.841216] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.101Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1231gitlab # [ 106.843409] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.103Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1232gitlab # [ 106.845651] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.105Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1233gitlab # [ 106.848095] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.108Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1234gitlab # [ 106.850827] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.111Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1235gitlab # [ 106.853198] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.113Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1236gitlab # [ 106.855823] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.116Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1237gitlab # [ 106.857920] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.118Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1238gitlab # [ 106.860670] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.120Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1239gitlab # [ 106.863115] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.123Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1240gitlab # [ 106.865114] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.125Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1241gitlab # [ 106.867520] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.127Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1242gitlab # [ 106.869839] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.130Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1243gitlab # [ 106.872094] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.132Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1244gitlab # [ 106.874380] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.134Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1245gitlab # [ 106.876589] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.136Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1246gitlab # [ 106.879359] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.139Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1247gitlab # [ 106.882273] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.142Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1248gitlab # [ 106.884661] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.144Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1249gitlab # [ 106.886857] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.147Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1250gitlab # [ 106.896594] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.156Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1251gitlab # [ 106.898950] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.159Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1252gitlab # [ 106.901275] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.161Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1253gitlab # [ 106.904413] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.164Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1254gitlab # [ 106.907067] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.167Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1255gitlab # [ 106.909887] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.170Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1256gitlab # [ 106.912889] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.173Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1257gitlab # [ 106.915096] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.175Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1258gitlab # [ 106.917080] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.177Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1259gitlab # [ 106.919887] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.180Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1260gitlab # [ 106.921907] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:24:42.182Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1261gitlab # [ 108.324062] gitlab-workhorse[833]: time="2026-09-08T04:24:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1262gitlab # [ 109.151446] sidekiq-cluster[1648]: InitializerConnections Query: SET client_min_messages TO 'warning'1263gitlab # [ 109.151603] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1264gitlab # [ 109.152417] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1265gitlab # [ 109.153229] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1266gitlab # [ 109.153967] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1267gitlab # [ 109.155808] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1268gitlab # [ 109.156912] sidekiq-cluster[1648]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1269gitlab # [ 109.176085] sidekiq-cluster[1648]: InitializerConnections Query: SET standard_conforming_strings = on1270gitlab # [ 109.176212] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1271gitlab # [ 109.176974] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1272gitlab # [ 109.178202] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1273gitlab # [ 109.178895] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1274gitlab # [ 109.180060] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1275gitlab # [ 109.180754] sidekiq-cluster[1648]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1276gitlab # [ 109.197660] sidekiq-cluster[1648]: InitializerConnections Query: SET intervalstyle = iso_86011277gitlab # [ 109.197824] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1278gitlab # [ 109.199075] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1279gitlab # [ 109.199948] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1280gitlab # [ 109.200311] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1281gitlab # [ 109.201242] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1282gitlab # [ 109.202169] sidekiq-cluster[1648]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1283gitlab # [ 109.212049] sidekiq-cluster[1648]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1284gitlab # [ 109.212314] sidekiq-cluster[1648]: FROM pg_type as t1285gitlab # [ 109.212739] sidekiq-cluster[1648]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1286gitlab # [ 109.213504] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1287gitlab # [ 109.214045] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1288gitlab # [ 109.214516] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1289gitlab # [ 109.215036] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1290gitlab # [ 109.215500] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1291gitlab # [ 109.216053] sidekiq-cluster[1648]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1292gitlab # [ 109.224819] sidekiq-cluster[1648]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1293gitlab # [ 109.224924] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1294gitlab # [ 109.225667] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1295gitlab # [ 109.226181] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1296gitlab # [ 109.226664] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1297gitlab # [ 109.227176] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1298gitlab # [ 109.227643] sidekiq-cluster[1648]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1299gitlab # [ 109.235493] sidekiq-cluster[1648]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1300gitlab # [ 109.235603] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1301gitlab # [ 109.236154] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1302gitlab # [ 109.236583] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1303gitlab # [ 109.237193] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1304gitlab # [ 109.237662] sidekiq-cluster[1648]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1305gitlab # [ 109.238202] sidekiq-cluster[1648]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1306gitlab # [ 118.334784] gitlab-workhorse[833]: time="2026-09-08T04:24:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1307gitlab # [ 128.336249] gitlab-workhorse[833]: time="2026-09-08T04:25:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1308gitlab # [ 130.162377] postgres[2095]: [2095] ERROR: relation "table_without_models" does not exist1309gitlab # [ 130.163378] postgres[2095]: [2095] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1310gitlab # [ 130.504875] postgres[2115]: [2115] ERROR: relation "table_without_models" does not exist1311gitlab # [ 130.505579] postgres[2115]: [2115] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1312gitlab # [ 132.633252] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:07.892Z","message":"Booted Rails 7.2.3.1 application in production environment"}1313gitlab # [ 132.636628] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:07.896Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1314gitlab # [ 132.639148] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:07.898Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1315gitlab # [ 132.640882] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:07.899Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1316gitlab # [ 132.643850] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:07.903Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1317gitlab # [ 133.213543] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.472Z","pid":1618,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1318gitlab # [ 133.214117] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.472Z","pid":1618,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1319gitlab # [ 133.214976] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.472Z","pid":1618,"message":"! #\u003cThread:0x0000748dda78eba0 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1320gitlab # [ 133.215719] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.473Z","pid":1618,"message":"! #\u003cThread:0x0000748dda9f4bd8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1321gitlab # [ 133.216677] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.473Z","pid":1618,"message":"Use Ctrl-C to stop"}1322gitlab # [ 133.217558] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.473Z","pid":1618,"message":"Sending status to systemd every 1.0 sec"}1323gitlab # [ 133.575497] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.835Z","pid":1618,"message":"- Worker 0 (PID: 2181) booted in 0.31s, phase: 0"}1324gitlab # [ 133.609044] bundle[1618]: {"timestamp":"2026-09-08T04:25:08.869Z","pid":1618,"message":"- Worker 1 (PID: 2185) booted in 0.33s, phase: 0"}1325gitlab # [ 133.615268] systemd[1]: Started gitlab.service.1326gitlab # [ 133.615973] systemd[1]: Reached target Common target for all GitLab services..1327gitlab # [ 133.619863] systemd[1]: Starting setup-gitlab.service...1328gitlab # [ 133.641910] setup-gitlab-start[2209]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1329gitlab # [ 134.153346] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:09.412Z","message":"Cleaning working queues"}1330gitlab # [ 134.153921] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:09.413Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"696fb2f3cda6444e3789ab25","created_at":"2026-09-08T04:24:10.754Z","trace_propagation_headers":{"sentry-trace":"8e8bf7d1425b4757ae2cd0a4bd85eeb5-e11c57143d4846bc","baggage":"sentry-trace_id=8e8bf7d1425b4757ae2cd0a4bd85eeb5,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e52d7a99ffafe5506e1aa93c8955630f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T04:24:10.766Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-696fb2f3cda6444e3789ab25: start","job_status":"start","queue_duration_s":58.647775,"scheduling_latency_s":58.647814}1331gitlab # [ 134.295059] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:09.554Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"696fb2f3cda6444e3789ab25","created_at":"2026-09-08T04:24:10.754Z","trace_propagation_headers":{"sentry-trace":"8e8bf7d1425b4757ae2cd0a4bd85eeb5-e11c57143d4846bc","baggage":"sentry-trace_id=8e8bf7d1425b4757ae2cd0a4bd85eeb5,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e52d7a99ffafe5506e1aa93c8955630f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T04:24:10.766Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-696fb2f3cda6444e3789ab25: done: 0.141 sec","job_status":"done","queue_duration_s":58.647775,"scheduling_latency_s":58.647814,"redis_calls":26,"redis_duration_s":0.006454,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003574,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001778,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07986462,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.141,"completed_at":"2026-09-08T04:25:09.554Z","load_balancing_strategy":"primary","db_duration_s":0.052013,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1332gitlab # [ 135.957137] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:25:11 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 22871333gitlab # [ 135.960886] setup-gitlab-start[2209]: + TOKEN=nixbot-test-token-of-20-chars1334gitlab # [ 135.961159] setup-gitlab-start[2209]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1335gitlab # [ 135.988864] runuser[2234]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1336gitlab # [ 136.343978] setup-gitlab-start[2235]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1337gitlab # [ 136.344375] setup-gitlab-start[2235]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1338gitlab # [ 136.345096] setup-gitlab-start[2235]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1339gitlab # [ 136.911996] setup-gitlab-start[2235]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1340gitlab # [ 136.912531] setup-gitlab-start[2235]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1341gitlab # [ 136.913956] setup-gitlab-start[2235]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1342gitlab # [ 145.216982] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"3309986d363eddc24f144989","created_at":"2026-09-08T04:25:20.440Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b0e35011c101c991159ca61f1568c4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.473Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-3309986d363eddc24f144989: start","job_status":"start","queue_duration_s":0.002789,"scheduling_latency_s":0.002824}1343gitlab # [ 145.330286] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"3309986d363eddc24f144989","created_at":"2026-09-08T04:25:20.440Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b0e35011c101c991159ca61f1568c4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.473Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-3309986d363eddc24f144989: done: 0.113561 sec","job_status":"done","queue_duration_s":0.002789,"scheduling_latency_s":0.002824,"redis_calls":12,"redis_duration_s":0.033815000000000005,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.026204,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003893,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001054324,"exclusive_lock_hold_duration_s":0.006043505,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.113561,"completed_at":"2026-09-08T04:25:20.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 145.383861] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"501ccdc2edfb928fafc7d2a0","created_at":"2026-09-08T04:25:20.513Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73519ddee02ef58488112e5e9d4584e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T04:25:20.529Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-501ccdc2edfb928fafc7d2a0: start","job_status":"start","queue_duration_s":0.114366,"scheduling_latency_s":0.114393}1345gitlab # [ 145.451677] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"501ccdc2edfb928fafc7d2a0","created_at":"2026-09-08T04:25:20.513Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73519ddee02ef58488112e5e9d4584e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T04:25:20.529Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-501ccdc2edfb928fafc7d2a0: done: 0.067806 sec","job_status":"done","queue_duration_s":0.114366,"scheduling_latency_s":0.114393,"redis_calls":10,"redis_duration_s":0.006054,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001905,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003236,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000412343,"duration_s":0.067806,"completed_at":"2026-09-08T04:25:20.711Z","load_balancing_strategy":"primary","db_duration_s":0.017851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 145.490746] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"00b7c9896d81e2cc98d9e993","created_at":"2026-09-08T04:25:20.563Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"662af43ebc78b9610c842c2ca1c5663a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.674Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-00b7c9896d81e2cc98d9e993: start","job_status":"start","queue_duration_s":0.076551,"scheduling_latency_s":0.076577}1347gitlab # [ 145.527990] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.787Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1348gitlab # [ 145.540517] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"00b7c9896d81e2cc98d9e993","created_at":"2026-09-08T04:25:20.563Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"662af43ebc78b9610c842c2ca1c5663a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.674Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-00b7c9896d81e2cc98d9e993: done: 0.049808 sec","job_status":"done","queue_duration_s":0.076551,"scheduling_latency_s":0.076577,"redis_calls":10,"redis_duration_s":0.009785,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0019,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000712661,"duration_s":0.049808,"completed_at":"2026-09-08T04:25:20.800Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 145.580449] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"aff56bae553c4636c9ca7a0b","created_at":"2026-09-08T04:25:20.694Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa0eb058b7dd7d9fc3e594a0ec1d73ff","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.706Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-aff56bae553c4636c9ca7a0b: start","job_status":"start","queue_duration_s":0.133667,"scheduling_latency_s":0.133694}1350gitlab # [ 145.656675] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"aff56bae553c4636c9ca7a0b","created_at":"2026-09-08T04:25:20.694Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa0eb058b7dd7d9fc3e594a0ec1d73ff","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.706Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-aff56bae553c4636c9ca7a0b: done: 0.076293 sec","job_status":"done","queue_duration_s":0.133667,"scheduling_latency_s":0.133694,"redis_calls":13,"redis_duration_s":0.012088999999999999,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001905,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007503,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002681,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015147176,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.076293,"completed_at":"2026-09-08T04:25:20.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011139,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1351gitlab # [ 145.662094] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:20.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"22da7b3c23ef6da0b86d58d5","created_at":"2026-09-08T04:25:20.748Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cce81628f0ec0c83cf618ff132f60fde","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.768Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-22da7b3c23ef6da0b86d58d5: start","job_status":"start","queue_duration_s":0.153647,"scheduling_latency_s":0.15367}1352gitlab # [ 145.744130] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"22da7b3c23ef6da0b86d58d5","created_at":"2026-09-08T04:25:20.748Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cce81628f0ec0c83cf618ff132f60fde","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.768Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-22da7b3c23ef6da0b86d58d5: done: 0.081436 sec","job_status":"done","queue_duration_s":0.153647,"scheduling_latency_s":0.15367,"redis_calls":10,"redis_duration_s":0.041901999999999995,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005762,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.030683,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000982806,"duration_s":0.081436,"completed_at":"2026-09-08T04:25:21.003Z","load_balancing_strategy":"primary","db_duration_s":0.007538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 145.789897] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"a3750da498f2a06301fef0c3","created_at":"2026-09-08T04:25:20.780Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b72c315a8b20677a34522f20193d9de4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.793Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-a3750da498f2a06301fef0c3: start","job_status":"start","queue_duration_s":0.256146,"scheduling_latency_s":0.256177}1354gitlab # [ 145.881214] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"a3750da498f2a06301fef0c3","created_at":"2026-09-08T04:25:20.780Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b72c315a8b20677a34522f20193d9de4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.793Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-a3750da498f2a06301fef0c3: done: 0.090966 sec","job_status":"done","queue_duration_s":0.256146,"scheduling_latency_s":0.256177,"redis_calls":17,"redis_duration_s":0.018274,"redis_read_bytes":15,"redis_write_bytes":2007,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.008944,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1150,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00165,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000964089,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.090966,"completed_at":"2026-09-08T04:25:21.140Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 145.920194] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0e6187307fbb17f12684f958","created_at":"2026-09-08T04:25:20.802Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d927d88ac9ed91bf56b8a8ad05e21201","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.806Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0e6187307fbb17f12684f958: start","job_status":"start","queue_duration_s":0.338124,"scheduling_latency_s":0.338157}1356gitlab # [ 145.920757] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.162Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1357gitlab # [ 145.921422] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.170Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0e6187307fbb17f12684f958","created_at":"2026-09-08T04:25:20.802Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d927d88ac9ed91bf56b8a8ad05e21201","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.806Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0e6187307fbb17f12684f958: done: 0.026253 sec","job_status":"done","queue_duration_s":0.338124,"scheduling_latency_s":0.338157,"redis_calls":10,"redis_duration_s":0.002704,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001552,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000582,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000377702,"duration_s":0.026253,"completed_at":"2026-09-08T04:25:21.170Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 145.977169] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f6afc89916c63aca24171c05","created_at":"2026-09-08T04:25:20.810Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.813Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f6afc89916c63aca24171c05: start","job_status":"start","queue_duration_s":0.358656,"scheduling_latency_s":0.358681}1359gitlab # [ 146.076891] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f6afc89916c63aca24171c05","created_at":"2026-09-08T04:25:20.810Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.813Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f6afc89916c63aca24171c05: done: 0.163796 sec","job_status":"done","queue_duration_s":0.358656,"scheduling_latency_s":0.358681,"redis_calls":19,"redis_duration_s":0.10502500000000001,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.085068,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000588,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.017882,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011346414,"duration_s":0.163796,"completed_at":"2026-09-08T04:25:21.336Z","load_balancing_strategy":"primary","db_duration_s":0.011853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 146.134780] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b3782a495afa2d357f2a47ec","created_at":"2026-09-08T04:25:20.823Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffc507d794136d839dd43ae37d2394b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.827Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b3782a495afa2d357f2a47ec: start","job_status":"start","queue_duration_s":0.511404,"scheduling_latency_s":0.511437}1361gitlab # [ 146.170517] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b3782a495afa2d357f2a47ec","created_at":"2026-09-08T04:25:20.823Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffc507d794136d839dd43ae37d2394b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.827Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b3782a495afa2d357f2a47ec: done: 0.091178 sec","job_status":"done","queue_duration_s":0.511404,"scheduling_latency_s":0.511437,"redis_calls":14,"redis_duration_s":0.00843,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001657,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000674387,"exclusive_lock_hold_duration_s":0.045320844,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.091178,"completed_at":"2026-09-08T04:25:21.429Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 146.190893] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.432Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a5cce79d0f984b3cdfcd041d","created_at":"2026-09-08T04:25:20.831Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ebdd786f42a98ba0c6221c2ae35c032","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.834Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a5cce79d0f984b3cdfcd041d: start","job_status":"start","queue_duration_s":0.597534,"scheduling_latency_s":0.597566}1363gitlab # [ 146.232302] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a5cce79d0f984b3cdfcd041d","created_at":"2026-09-08T04:25:20.831Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ebdd786f42a98ba0c6221c2ae35c032","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.834Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a5cce79d0f984b3cdfcd041d: done: 0.059415 sec","job_status":"done","queue_duration_s":0.597534,"scheduling_latency_s":0.597566,"redis_calls":12,"redis_duration_s":0.002698,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000992,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000268749,"duration_s":0.059415,"completed_at":"2026-09-08T04:25:21.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 146.233804] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b5b0bc5c7450d84ad976384e","created_at":"2026-09-08T04:25:20.838Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9fcbddad015ba77215b8260228760d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T04:25:20.858Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b5b0bc5c7450d84ad976384e: start","job_status":"start","queue_duration_s":0.634265,"scheduling_latency_s":0.634279}1365gitlab # [ 146.285751] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.545Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1366gitlab # [ 146.298654] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b5b0bc5c7450d84ad976384e","created_at":"2026-09-08T04:25:20.838Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9fcbddad015ba77215b8260228760d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T04:25:20.858Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b5b0bc5c7450d84ad976384e: done: 0.06543 sec","job_status":"done","queue_duration_s":0.634265,"scheduling_latency_s":0.634279,"redis_calls":10,"redis_duration_s":0.00237,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000545,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000575213,"duration_s":0.06543,"completed_at":"2026-09-08T04:25:21.557Z","load_balancing_strategy":"primary","db_duration_s":0.042453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 146.336217] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b0e50a08c0b0248884c8c94d","created_at":"2026-09-08T04:25:20.869Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be6f673ceb1778f68d810e4dc490a281","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.877Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b0e50a08c0b0248884c8c94d: start","job_status":"start","queue_duration_s":0.717961,"scheduling_latency_s":0.718002}1368gitlab # [ 146.662316] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b0e50a08c0b0248884c8c94d","created_at":"2026-09-08T04:25:20.869Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be6f673ceb1778f68d810e4dc490a281","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.877Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b0e50a08c0b0248884c8c94d: done: 0.326517 sec","job_status":"done","queue_duration_s":0.717961,"scheduling_latency_s":0.718002,"redis_calls":565,"redis_duration_s":0.09201000000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00112,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.090452,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.250481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021567,"duration_s":0.326517,"completed_at":"2026-09-08T04:25:21.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 146.701260] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:21.961Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8a268dd7d6700a7413c59dfe","created_at":"2026-09-08T04:25:20.890Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aaff0351ed94a0f1d0c6aa3933e10880","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.897Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8a268dd7d6700a7413c59dfe: start","job_status":"start","queue_duration_s":1.063407,"scheduling_latency_s":1.063433}1370gitlab # [ 146.908676] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8a268dd7d6700a7413c59dfe","created_at":"2026-09-08T04:25:20.890Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aaff0351ed94a0f1d0c6aa3933e10880","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.897Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8a268dd7d6700a7413c59dfe: done: 0.20745 sec","job_status":"done","queue_duration_s":1.063407,"scheduling_latency_s":1.063433,"redis_calls":12,"redis_duration_s":0.00297,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001381,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000474362,"exclusive_lock_hold_duration_s":0.17683979400000002,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.20745,"completed_at":"2026-09-08T04:25:22.168Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 146.949643] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c1b9e0f9949867ed7c4f80b1","created_at":"2026-09-08T04:25:20.908Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f52cdc6bf9e53768afa39d220a182070","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.920Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c1b9e0f9949867ed7c4f80b1: start","job_status":"start","queue_duration_s":1.289398,"scheduling_latency_s":1.289425}1372gitlab # [ 146.993848] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c1b9e0f9949867ed7c4f80b1","created_at":"2026-09-08T04:25:20.908Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f52cdc6bf9e53768afa39d220a182070","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.920Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c1b9e0f9949867ed7c4f80b1: done: 0.043806 sec","job_status":"done","queue_duration_s":1.289398,"scheduling_latency_s":1.289425,"redis_calls":10,"redis_duration_s":0.003444,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000708,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002002,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000570184,"duration_s":0.043806,"completed_at":"2026-09-08T04:25:22.253Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 147.016530] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.255Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4f30a0de7eff6a2c01e79bad","created_at":"2026-09-08T04:25:20.925Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.933Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4f30a0de7eff6a2c01e79bad: start","job_status":"start","queue_duration_s":1.321981,"scheduling_latency_s":1.322015}1374gitlab # [ 147.135080] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4f30a0de7eff6a2c01e79bad","created_at":"2026-09-08T04:25:20.925Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.933Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4f30a0de7eff6a2c01e79bad: done: 0.139117 sec","job_status":"done","queue_duration_s":1.321981,"scheduling_latency_s":1.322015,"redis_calls":71,"redis_duration_s":0.019557,"redis_read_bytes":120,"redis_write_bytes":42524,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003613,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001086,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37335,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013988,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310654,"duration_s":0.139117,"completed_at":"2026-09-08T04:25:22.394Z","load_balancing_strategy":"primary","db_duration_s":0.027641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 147.135975] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"52367ee18279c170096c5025","created_at":"2026-09-08T04:25:20.956Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"498cd330273a35fa1c896ad4d2337f5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T04:25:20.963Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-52367ee18279c170096c5025: start","job_status":"start","queue_duration_s":1.432719,"scheduling_latency_s":1.432734}1376gitlab # [ 147.221405] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.481Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1377gitlab # [ 147.235469] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"52367ee18279c170096c5025","created_at":"2026-09-08T04:25:20.956Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"498cd330273a35fa1c896ad4d2337f5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T04:25:20.963Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-52367ee18279c170096c5025: done: 0.097874 sec","job_status":"done","queue_duration_s":1.432719,"scheduling_latency_s":1.432734,"redis_calls":10,"redis_duration_s":0.0047550000000000005,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003768,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.075,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310374,"duration_s":0.097874,"completed_at":"2026-09-08T04:25:22.493Z","load_balancing_strategy":"primary","db_duration_s":0.075309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 147.273832] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"72800d2063c0fab33bb47aef","created_at":"2026-09-08T04:25:20.969Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb45dcece474b114c5f9abc202566c61","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.979Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-72800d2063c0fab33bb47aef: start","job_status":"start","queue_duration_s":1.553739,"scheduling_latency_s":1.553781}1379gitlab # [ 147.357869] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"72800d2063c0fab33bb47aef","created_at":"2026-09-08T04:25:20.969Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb45dcece474b114c5f9abc202566c61","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:20.979Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-72800d2063c0fab33bb47aef: done: 0.083533 sec","job_status":"done","queue_duration_s":1.553739,"scheduling_latency_s":1.553781,"redis_calls":12,"redis_duration_s":0.003921,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001134223,"exclusive_lock_hold_duration_s":0.019667305,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.083533,"completed_at":"2026-09-08T04:25:22.616Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 147.430400] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ea35b08f92daf678c65ef776","created_at":"2026-09-08T04:25:20.994Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a4790ab9ca06257ae6299fb8649e33f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.049Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ea35b08f92daf678c65ef776: start","job_status":"start","queue_duration_s":1.640832,"scheduling_latency_s":1.640881}1381gitlab # [ 147.492067] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ea35b08f92daf678c65ef776","created_at":"2026-09-08T04:25:20.994Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a4790ab9ca06257ae6299fb8649e33f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.049Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ea35b08f92daf678c65ef776: done: 0.061118 sec","job_status":"done","queue_duration_s":1.640832,"scheduling_latency_s":1.640881,"redis_calls":10,"redis_duration_s":0.003764,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000756,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00071,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000625778,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.061118,"completed_at":"2026-09-08T04:25:22.750Z","load_balancing_strategy":"primary","db_duration_s":0.020315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 147.537524] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0c58502541e874ae01ba802a","created_at":"2026-09-08T04:25:21.079Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bde1e371dbba8a34484a9117ca014dd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.098Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0c58502541e874ae01ba802a: start","job_status":"start","queue_duration_s":1.698638,"scheduling_latency_s":1.698689}1383gitlab # [ 147.576792] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0c58502541e874ae01ba802a","created_at":"2026-09-08T04:25:21.079Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bde1e371dbba8a34484a9117ca014dd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.098Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0c58502541e874ae01ba802a: done: 0.03948 sec","job_status":"done","queue_duration_s":1.698638,"scheduling_latency_s":1.698689,"redis_calls":10,"redis_duration_s":0.003502,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00065,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002199,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000547555,"duration_s":0.03948,"completed_at":"2026-09-08T04:25:22.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 147.617956] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"84c7c6cd3eceacd7900a3760","created_at":"2026-09-08T04:25:21.109Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c58184497f873d2bc2384453dab3477b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.120Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-84c7c6cd3eceacd7900a3760: start","job_status":"start","queue_duration_s":1.7575,"scheduling_latency_s":1.757534}1385gitlab # [ 147.653603] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.913Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1386gitlab # [ 147.666479] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.925Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"84c7c6cd3eceacd7900a3760","created_at":"2026-09-08T04:25:21.109Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c58184497f873d2bc2384453dab3477b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.120Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-84c7c6cd3eceacd7900a3760: done: 0.04822 sec","job_status":"done","queue_duration_s":1.7575,"scheduling_latency_s":1.757534,"redis_calls":10,"redis_duration_s":0.003438,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00067,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001863,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000482184,"duration_s":0.04822,"completed_at":"2026-09-08T04:25:22.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 147.706188] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:22.965Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"55ca23155260614fc27b908d","created_at":"2026-09-08T04:25:21.136Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d58e115f90338cedf576a814b54c7346","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.269Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-55ca23155260614fc27b908d: start","job_status":"start","queue_duration_s":1.69589,"scheduling_latency_s":1.695927}1388gitlab # [ 147.758094] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"55ca23155260614fc27b908d","created_at":"2026-09-08T04:25:21.136Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d58e115f90338cedf576a814b54c7346","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.269Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-55ca23155260614fc27b908d: done: 0.051719 sec","job_status":"done","queue_duration_s":1.69589,"scheduling_latency_s":1.695927,"redis_calls":12,"redis_duration_s":0.002787,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000753447,"exclusive_lock_hold_duration_s":0.01023929,"duration_s":0.051719,"completed_at":"2026-09-08T04:25:23.017Z","load_balancing_strategy":"primary","db_duration_s":0.005166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 147.795812] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.055Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"4bfdb8e3a905edda8ce040b1","created_at":"2026-09-08T04:25:21.281Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9a4c34a9de77e3196d308710c48f3a5","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.303Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-4bfdb8e3a905edda8ce040b1: start","job_status":"start","queue_duration_s":1.751718,"scheduling_latency_s":1.751762}1390gitlab # [ 147.844978] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.103Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"4bfdb8e3a905edda8ce040b1","created_at":"2026-09-08T04:25:21.281Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9a4c34a9de77e3196d308710c48f3a5","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.303Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-4bfdb8e3a905edda8ce040b1: done: 0.048659 sec","job_status":"done","queue_duration_s":1.751718,"scheduling_latency_s":1.751762,"redis_calls":10,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":9,"redis_write_bytes":1109,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001441,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000484419,"duration_s":0.048659,"completed_at":"2026-09-08T04:25:23.103Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 147.883539] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.143Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55ec73905e6feb7b9f7f35f3","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55ec73905e6feb7b9f7f35f3: start","job_status":"start","queue_duration_s":1.836101,"scheduling_latency_s":1.836137}1392gitlab # [ 147.918774] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.178Z","message":"No existing merge request to be cleaned up."}1393gitlab # [ 147.939558] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.198Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55ec73905e6feb7b9f7f35f3","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55ec73905e6feb7b9f7f35f3: done: 0.055981 sec","job_status":"done","queue_duration_s":1.836101,"scheduling_latency_s":1.836137,"redis_calls":13,"redis_duration_s":0.004641999999999999,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000415695,"duration_s":0.055981,"completed_at":"2026-09-08T04:25:23.198Z","load_balancing_strategy":"primary","db_duration_s":0.006845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 147.977111] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.236Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cf2ff4d346d4ee946b3e3ea1","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cf2ff4d346d4ee946b3e3ea1: start","job_status":"start","queue_duration_s":1.929997,"scheduling_latency_s":1.93003}1395gitlab # [ 147.992908] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.252Z","message":"No existing merge request to be cleaned up."}1396gitlab # [ 148.004914] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.264Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cf2ff4d346d4ee946b3e3ea1","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cf2ff4d346d4ee946b3e3ea1: done: 0.027482 sec","job_status":"done","queue_duration_s":1.929997,"scheduling_latency_s":1.93003,"redis_calls":10,"redis_duration_s":0.00353,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001332,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002198,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027482,"completed_at":"2026-09-08T04:25:23.264Z","load_balancing_strategy":"primary","db_duration_s":0.002455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 148.027353] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.266Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"69d451945b37b52dcc013413","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-69d451945b37b52dcc013413: start","job_status":"start","queue_duration_s":1.959686,"scheduling_latency_s":1.959713}1398gitlab # [ 148.028451] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.276Z","message":"No existing merge request to be cleaned up."}1399gitlab # [ 148.029531] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.282Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"69d451945b37b52dcc013413","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-69d451945b37b52dcc013413: done: 0.015877 sec","job_status":"done","queue_duration_s":1.959686,"scheduling_latency_s":1.959713,"redis_calls":10,"redis_duration_s":0.0023420000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015877,"completed_at":"2026-09-08T04:25:23.282Z","load_balancing_strategy":"primary","db_duration_s":0.001757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 148.074367] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.283Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38d69613bfc995b4932eccd5","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38d69613bfc995b4932eccd5: start","job_status":"start","queue_duration_s":1.976701,"scheduling_latency_s":1.976716}1401gitlab # [ 148.100503] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.360Z","message":"No existing merge request to be cleaned up."}1402gitlab # [ 148.116152] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.375Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38d69613bfc995b4932eccd5","created_at":"2026-09-08T04:25:21.273Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-9a37ef73126043ed","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70e8bbcffa1f46b34ebe7b0f3d8b8238","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.306Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38d69613bfc995b4932eccd5: done: 0.091947 sec","job_status":"done","queue_duration_s":1.976701,"scheduling_latency_s":1.976716,"redis_calls":10,"redis_duration_s":0.004903,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003454,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.068,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091947,"completed_at":"2026-09-08T04:25:23.375Z","load_balancing_strategy":"primary","db_duration_s":0.135417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 148.151898] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8f9e99d376ecc280cebecec5","created_at":"2026-09-08T04:25:21.323Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"666ae02e61e6fd61ef0ff8951b34974a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.343Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8f9e99d376ecc280cebecec5: start","job_status":"start","queue_duration_s":2.067728,"scheduling_latency_s":2.067774}1404gitlab # [ 148.181603] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.441Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1405gitlab # [ 148.195829] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8f9e99d376ecc280cebecec5","created_at":"2026-09-08T04:25:21.323Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"666ae02e61e6fd61ef0ff8951b34974a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T04:25:21.343Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8f9e99d376ecc280cebecec5: done: 0.043404 sec","job_status":"done","queue_duration_s":2.067728,"scheduling_latency_s":2.067774,"redis_calls":10,"redis_duration_s":0.004276,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003334,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000467937,"duration_s":0.043404,"completed_at":"2026-09-08T04:25:23.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 148.235455] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaafcb9b2d7c73d4aecc7c1e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaafcb9b2d7c73d4aecc7c1e: start","job_status":"start","queue_duration_s":1.107277,"scheduling_latency_s":1.10732}1407gitlab # [ 148.270544] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaafcb9b2d7c73d4aecc7c1e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaafcb9b2d7c73d4aecc7c1e: done: 0.035966 sec","job_status":"done","queue_duration_s":1.107277,"scheduling_latency_s":1.10732,"redis_calls":16,"redis_duration_s":0.004834,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000656,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002784,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000446705,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035966,"completed_at":"2026-09-08T04:25:23.530Z","load_balancing_strategy":"primary","db_duration_s":0.000679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 148.310236] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9cfee891d27364801a23160","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9cfee891d27364801a23160: start","job_status":"start","queue_duration_s":1.18281,"scheduling_latency_s":1.182851}1409gitlab # [ 148.335993] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9cfee891d27364801a23160","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9cfee891d27364801a23160: done: 0.025976 sec","job_status":"done","queue_duration_s":1.18281,"scheduling_latency_s":1.182851,"redis_calls":13,"redis_duration_s":0.003962,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001378,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002584,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025976,"completed_at":"2026-09-08T04:25:23.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 148.373050] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfa8f41088646c7fbc4aa0c9","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfa8f41088646c7fbc4aa0c9: start","job_status":"start","queue_duration_s":1.245779,"scheduling_latency_s":1.24582}1411gitlab # [ 148.397476] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfa8f41088646c7fbc4aa0c9","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfa8f41088646c7fbc4aa0c9: done: 0.02431 sec","job_status":"done","queue_duration_s":1.245779,"scheduling_latency_s":1.24582,"redis_calls":13,"redis_duration_s":0.003613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02431,"completed_at":"2026-09-08T04:25:23.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 148.434644] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9299ffc574b8fec4ad448853","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9299ffc574b8fec4ad448853: start","job_status":"start","queue_duration_s":1.306926,"scheduling_latency_s":1.306965}1413gitlab # [ 148.467313] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9299ffc574b8fec4ad448853","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9299ffc574b8fec4ad448853: done: 0.032872 sec","job_status":"done","queue_duration_s":1.306926,"scheduling_latency_s":1.306965,"redis_calls":13,"redis_duration_s":0.002911,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032872,"completed_at":"2026-09-08T04:25:23.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 148.508169] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47282b709c4df4431d6d3b66","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47282b709c4df4431d6d3b66: start","job_status":"start","queue_duration_s":1.378959,"scheduling_latency_s":1.378999}1415gitlab # [ 148.545591] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47282b709c4df4431d6d3b66","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47282b709c4df4431d6d3b66: done: 0.039233 sec","job_status":"done","queue_duration_s":1.378959,"scheduling_latency_s":1.378999,"redis_calls":13,"redis_duration_s":0.004000999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039233,"completed_at":"2026-09-08T04:25:23.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 148.582729] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cc80f202ae60f55e9a998e2","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cc80f202ae60f55e9a998e2: start","job_status":"start","queue_duration_s":1.455231,"scheduling_latency_s":1.455276}1417gitlab # [ 148.609109] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cc80f202ae60f55e9a998e2","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cc80f202ae60f55e9a998e2: done: 0.02654 sec","job_status":"done","queue_duration_s":1.455231,"scheduling_latency_s":1.455276,"redis_calls":13,"redis_duration_s":0.004243,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002751,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02654,"completed_at":"2026-09-08T04:25:23.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 148.612448] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e69e54a9be9a1d6e506f688","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e69e54a9be9a1d6e506f688: start","job_status":"start","queue_duration_s":1.485241,"scheduling_latency_s":1.485275}1419gitlab # [ 148.635748] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e69e54a9be9a1d6e506f688","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e69e54a9be9a1d6e506f688: done: 0.023204 sec","job_status":"done","queue_duration_s":1.485241,"scheduling_latency_s":1.485275,"redis_calls":13,"redis_duration_s":0.003571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023204,"completed_at":"2026-09-08T04:25:23.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 148.652302] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9db2c4a1e1655dc70e201b5","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9db2c4a1e1655dc70e201b5: start","job_status":"start","queue_duration_s":1.51478,"scheduling_latency_s":1.514807}1421gitlab # [ 148.703057] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9db2c4a1e1655dc70e201b5","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9db2c4a1e1655dc70e201b5: done: 0.024223 sec","job_status":"done","queue_duration_s":1.51478,"scheduling_latency_s":1.514807,"redis_calls":13,"redis_duration_s":0.008196,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024223,"completed_at":"2026-09-08T04:25:23.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 148.704519] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c54df1f4a12c775c084ae6f","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c54df1f4a12c775c084ae6f: start","job_status":"start","queue_duration_s":1.540169,"scheduling_latency_s":1.540183}1423gitlab # [ 148.712659] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c54df1f4a12c775c084ae6f","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c54df1f4a12c775c084ae6f: done: 0.011265 sec","job_status":"done","queue_duration_s":1.540169,"scheduling_latency_s":1.540183,"redis_calls":13,"redis_duration_s":0.002802,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011265,"completed_at":"2026-09-08T04:25:23.938Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 148.755701] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f23cd41ebd769c2ef0381ae","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f23cd41ebd769c2ef0381ae: start","job_status":"start","queue_duration_s":1.552674,"scheduling_latency_s":1.552688}1425gitlab # [ 148.762298] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f23cd41ebd769c2ef0381ae","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f23cd41ebd769c2ef0381ae: done: 0.011048 sec","job_status":"done","queue_duration_s":1.552674,"scheduling_latency_s":1.552688,"redis_calls":13,"redis_duration_s":0.002563,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011048,"completed_at":"2026-09-08T04:25:23.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 148.817713] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78aed5b4265e6f60e096a256","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78aed5b4265e6f60e096a256: start","job_status":"start","queue_duration_s":1.564962,"scheduling_latency_s":1.564976}1427gitlab # [ 148.818567] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78aed5b4265e6f60e096a256","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78aed5b4265e6f60e096a256: done: 0.0108 sec","job_status":"done","queue_duration_s":1.564962,"scheduling_latency_s":1.564976,"redis_calls":13,"redis_duration_s":0.002507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0108,"completed_at":"2026-09-08T04:25:23.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 148.862140] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5305f0841b00d929ed78fb28","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5305f0841b00d929ed78fb28: start","job_status":"start","queue_duration_s":1.581687,"scheduling_latency_s":1.581703}1429gitlab # [ 148.862890] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5305f0841b00d929ed78fb28","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5305f0841b00d929ed78fb28: done: 0.013015 sec","job_status":"done","queue_duration_s":1.581687,"scheduling_latency_s":1.581703,"redis_calls":13,"redis_duration_s":0.0036680000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00172,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013015,"completed_at":"2026-09-08T04:25:23.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 148.887214] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fdf0b52255de81a130d4efc","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fdf0b52255de81a130d4efc: start","job_status":"start","queue_duration_s":1.595918,"scheduling_latency_s":1.595932}1431gitlab # [ 148.931814] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fdf0b52255de81a130d4efc","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fdf0b52255de81a130d4efc: done: 0.01087 sec","job_status":"done","queue_duration_s":1.595918,"scheduling_latency_s":1.595932,"redis_calls":13,"redis_duration_s":0.002664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01087,"completed_at":"2026-09-08T04:25:23.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 148.933169] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:23.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdc02ae220e3e88f0e840850","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdc02ae220e3e88f0e840850: start","job_status":"start","queue_duration_s":1.608503,"scheduling_latency_s":1.608517}1433gitlab # [ 148.980810] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdc02ae220e3e88f0e840850","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdc02ae220e3e88f0e840850: done: 0.010731 sec","job_status":"done","queue_duration_s":1.608503,"scheduling_latency_s":1.608517,"redis_calls":13,"redis_duration_s":0.002569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001724,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010731,"completed_at":"2026-09-08T04:25:24.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 148.982026] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e833ac04b6f4a2e99dc35708","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e833ac04b6f4a2e99dc35708: start","job_status":"start","queue_duration_s":1.620404,"scheduling_latency_s":1.620417}1435gitlab # [ 149.031231] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e833ac04b6f4a2e99dc35708","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e833ac04b6f4a2e99dc35708: done: 0.014843 sec","job_status":"done","queue_duration_s":1.620404,"scheduling_latency_s":1.620417,"redis_calls":13,"redis_duration_s":0.006366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00328,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014843,"completed_at":"2026-09-08T04:25:24.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 149.032437] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e525daf7137b97e6f42bb6ca","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e525daf7137b97e6f42bb6ca: start","job_status":"start","queue_duration_s":1.63852,"scheduling_latency_s":1.638568}1437gitlab # [ 149.039502] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e525daf7137b97e6f42bb6ca","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e525daf7137b97e6f42bb6ca: done: 0.023756 sec","job_status":"done","queue_duration_s":1.63852,"scheduling_latency_s":1.638568,"redis_calls":13,"redis_duration_s":0.00297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023756,"completed_at":"2026-09-08T04:25:24.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 149.105814] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6981fb6a0ef2c19977b033cf","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6981fb6a0ef2c19977b033cf: start","job_status":"start","queue_duration_s":1.663616,"scheduling_latency_s":1.663641}1439gitlab # [ 149.106639] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6981fb6a0ef2c19977b033cf","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6981fb6a0ef2c19977b033cf: done: 0.016484 sec","job_status":"done","queue_duration_s":1.663616,"scheduling_latency_s":1.663641,"redis_calls":13,"redis_duration_s":0.002129,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016484,"completed_at":"2026-09-08T04:25:24.066Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 149.148463] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffc47c6612390a96ff1c7e5c","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffc47c6612390a96ff1c7e5c: start","job_status":"start","queue_duration_s":1.680932,"scheduling_latency_s":1.680945}1441gitlab # [ 149.149275] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffc47c6612390a96ff1c7e5c","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffc47c6612390a96ff1c7e5c: done: 0.009503 sec","job_status":"done","queue_duration_s":1.680932,"scheduling_latency_s":1.680945,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009503,"completed_at":"2026-09-08T04:25:24.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 149.192131] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc236df4ae6f6539e64f3123","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc236df4ae6f6539e64f3123: start","job_status":"start","queue_duration_s":1.695687,"scheduling_latency_s":1.695701}1443gitlab # [ 149.192789] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc236df4ae6f6539e64f3123","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc236df4ae6f6539e64f3123: done: 0.00988 sec","job_status":"done","queue_duration_s":1.695687,"scheduling_latency_s":1.695701,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00988,"completed_at":"2026-09-08T04:25:24.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 149.247026] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67fbd0c50dd89e1366d14214","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67fbd0c50dd89e1366d14214: start","job_status":"start","queue_duration_s":1.706576,"scheduling_latency_s":1.706589}1445gitlab # [ 149.247594] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67fbd0c50dd89e1366d14214","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67fbd0c50dd89e1366d14214: done: 0.009725 sec","job_status":"done","queue_duration_s":1.706576,"scheduling_latency_s":1.706589,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009725,"completed_at":"2026-09-08T04:25:24.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 149.291088] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97ea97334e83a4f9f00e7533","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97ea97334e83a4f9f00e7533: start","job_status":"start","queue_duration_s":1.71749,"scheduling_latency_s":1.717504}1447gitlab # [ 149.291744] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97ea97334e83a4f9f00e7533","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97ea97334e83a4f9f00e7533: done: 0.060994 sec","job_status":"done","queue_duration_s":1.71749,"scheduling_latency_s":1.717504,"redis_calls":13,"redis_duration_s":0.001666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060994,"completed_at":"2026-09-08T04:25:24.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 149.330601] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3c909ccbf0a0d478f13d07e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3c909ccbf0a0d478f13d07e: start","job_status":"start","queue_duration_s":1.779511,"scheduling_latency_s":1.77953}1449gitlab # [ 149.331457] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3c909ccbf0a0d478f13d07e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3c909ccbf0a0d478f13d07e: done: 0.009875 sec","job_status":"done","queue_duration_s":1.779511,"scheduling_latency_s":1.77953,"redis_calls":13,"redis_duration_s":0.001477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009875,"completed_at":"2026-09-08T04:25:24.176Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 149.387096] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cbd34c7215760a2cd434b93","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cbd34c7215760a2cd434b93: start","job_status":"start","queue_duration_s":1.790259,"scheduling_latency_s":1.790272}1451gitlab # [ 149.387743] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cbd34c7215760a2cd434b93","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cbd34c7215760a2cd434b93: done: 0.008889 sec","job_status":"done","queue_duration_s":1.790259,"scheduling_latency_s":1.790272,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008889,"completed_at":"2026-09-08T04:25:24.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 149.397739] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2854d60c25bcb49b3a527f35","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2854d60c25bcb49b3a527f35: start","job_status":"start","queue_duration_s":1.800043,"scheduling_latency_s":1.800056}1453gitlab # [ 149.443142] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2854d60c25bcb49b3a527f35","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2854d60c25bcb49b3a527f35: done: 0.017724 sec","job_status":"done","queue_duration_s":1.800043,"scheduling_latency_s":1.800056,"redis_calls":13,"redis_duration_s":0.009673999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017724,"completed_at":"2026-09-08T04:25:24.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 149.444078] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"771fb46b29d3c597341097df","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-771fb46b29d3c597341097df: start","job_status":"start","queue_duration_s":1.818612,"scheduling_latency_s":1.818626}1455gitlab # [ 149.482047] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"771fb46b29d3c597341097df","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-771fb46b29d3c597341097df: done: 0.009117 sec","job_status":"done","queue_duration_s":1.818612,"scheduling_latency_s":1.818626,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009117,"completed_at":"2026-09-08T04:25:24.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 149.486847] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c926ed5db871797db2dea629","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c926ed5db871797db2dea629: start","job_status":"start","queue_duration_s":1.82858,"scheduling_latency_s":1.828594}1457gitlab # [ 149.529533] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c926ed5db871797db2dea629","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c926ed5db871797db2dea629: done: 0.009102 sec","job_status":"done","queue_duration_s":1.82858,"scheduling_latency_s":1.828594,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009102,"completed_at":"2026-09-08T04:25:24.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 149.533525] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40e88dbbd50efc87c1e0d5e7","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40e88dbbd50efc87c1e0d5e7: start","job_status":"start","queue_duration_s":1.838494,"scheduling_latency_s":1.838507}1459gitlab # [ 149.571646] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40e88dbbd50efc87c1e0d5e7","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40e88dbbd50efc87c1e0d5e7: done: 0.008892 sec","job_status":"done","queue_duration_s":1.838494,"scheduling_latency_s":1.838507,"redis_calls":13,"redis_duration_s":0.0014570000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008892,"completed_at":"2026-09-08T04:25:24.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 149.578019] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397a71880f5978d9ddaa47de","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397a71880f5978d9ddaa47de: start","job_status":"start","queue_duration_s":1.84818,"scheduling_latency_s":1.848193}1461gitlab # [ 149.589646] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397a71880f5978d9ddaa47de","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397a71880f5978d9ddaa47de: done: 0.017068 sec","job_status":"done","queue_duration_s":1.84818,"scheduling_latency_s":1.848193,"redis_calls":13,"redis_duration_s":0.008878,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001365,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017068,"completed_at":"2026-09-08T04:25:24.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 149.629928] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4281f841969b571325b00bf","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4281f841969b571325b00bf: start","job_status":"start","queue_duration_s":1.866095,"scheduling_latency_s":1.866108}1463gitlab # [ 149.634804] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4281f841969b571325b00bf","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4281f841969b571325b00bf: done: 0.009197 sec","job_status":"done","queue_duration_s":1.866095,"scheduling_latency_s":1.866108,"redis_calls":13,"redis_duration_s":0.001478,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009197,"completed_at":"2026-09-08T04:25:24.262Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 149.703213] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3a6344eb33bc792905ae2ed","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3a6344eb33bc792905ae2ed: start","job_status":"start","queue_duration_s":1.87609,"scheduling_latency_s":1.876103}1465gitlab # [ 149.703995] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3a6344eb33bc792905ae2ed","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3a6344eb33bc792905ae2ed: done: 0.008834 sec","job_status":"done","queue_duration_s":1.87609,"scheduling_latency_s":1.876103,"redis_calls":13,"redis_duration_s":0.0014449999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008834,"completed_at":"2026-09-08T04:25:24.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 149.714597] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0923eed93cc46eaafd8a5740","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0923eed93cc46eaafd8a5740: start","job_status":"start","queue_duration_s":1.885753,"scheduling_latency_s":1.885766}1467gitlab # [ 149.725595] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0923eed93cc46eaafd8a5740","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0923eed93cc46eaafd8a5740: done: 0.00894 sec","job_status":"done","queue_duration_s":1.885753,"scheduling_latency_s":1.885766,"redis_calls":13,"redis_duration_s":0.001455,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00894,"completed_at":"2026-09-08T04:25:24.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 149.774576] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2804bd898a75f8878c98f63c","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2804bd898a75f8878c98f63c: start","job_status":"start","queue_duration_s":1.89595,"scheduling_latency_s":1.895963}1469gitlab # [ 149.775510] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2804bd898a75f8878c98f63c","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2804bd898a75f8878c98f63c: done: 0.008957 sec","job_status":"done","queue_duration_s":1.89595,"scheduling_latency_s":1.895963,"redis_calls":13,"redis_duration_s":0.001913,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008957,"completed_at":"2026-09-08T04:25:24.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 149.825511] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5bab039384cff38e9db623e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5bab039384cff38e9db623e: start","job_status":"start","queue_duration_s":1.909624,"scheduling_latency_s":1.90964}1471gitlab # [ 149.826131] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5bab039384cff38e9db623e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5bab039384cff38e9db623e: done: 0.018142 sec","job_status":"done","queue_duration_s":1.909624,"scheduling_latency_s":1.90964,"redis_calls":13,"redis_duration_s":0.009129,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018142,"completed_at":"2026-09-08T04:25:24.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 149.867967] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd1bc8cdba19c194112b8cc8","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd1bc8cdba19c194112b8cc8: start","job_status":"start","queue_duration_s":1.930722,"scheduling_latency_s":1.930736}1473gitlab # [ 149.868665] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd1bc8cdba19c194112b8cc8","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd1bc8cdba19c194112b8cc8: done: 0.012664 sec","job_status":"done","queue_duration_s":1.930722,"scheduling_latency_s":1.930736,"redis_calls":13,"redis_duration_s":0.0041329999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002565,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012664,"completed_at":"2026-09-08T04:25:24.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 149.915609] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6af5d4d9bdbe618b05a13486","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6af5d4d9bdbe618b05a13486: start","job_status":"start","queue_duration_s":1.944272,"scheduling_latency_s":1.944285}1475gitlab # [ 149.916254] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6af5d4d9bdbe618b05a13486","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6af5d4d9bdbe618b05a13486: done: 0.008885 sec","job_status":"done","queue_duration_s":1.944272,"scheduling_latency_s":1.944285,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008885,"completed_at":"2026-09-08T04:25:24.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 149.961899] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56707e166b48656c627e2f49","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56707e166b48656c627e2f49: start","job_status":"start","queue_duration_s":1.95443,"scheduling_latency_s":1.954443}1477gitlab # [ 149.962658] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56707e166b48656c627e2f49","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56707e166b48656c627e2f49: done: 0.008896 sec","job_status":"done","queue_duration_s":1.95443,"scheduling_latency_s":1.954443,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008896,"completed_at":"2026-09-08T04:25:24.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 150.015306] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"881a70062c99ba7f9c94a28d","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-881a70062c99ba7f9c94a28d: start","job_status":"start","queue_duration_s":1.964136,"scheduling_latency_s":1.96415}1479gitlab # [ 150.015997] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"881a70062c99ba7f9c94a28d","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-881a70062c99ba7f9c94a28d: done: 0.008864 sec","job_status":"done","queue_duration_s":1.964136,"scheduling_latency_s":1.96415,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008864,"completed_at":"2026-09-08T04:25:24.359Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 150.027847] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44fca045e679a86887f58bc2","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44fca045e679a86887f58bc2: start","job_status":"start","queue_duration_s":1.97383,"scheduling_latency_s":1.973843}1481gitlab # [ 150.067327] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44fca045e679a86887f58bc2","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44fca045e679a86887f58bc2: done: 0.012695 sec","job_status":"done","queue_duration_s":1.97383,"scheduling_latency_s":1.973843,"redis_calls":13,"redis_duration_s":0.0047810000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012695,"completed_at":"2026-09-08T04:25:24.373Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 150.073186] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"291feefd8ba19d2b79c4355e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-291feefd8ba19d2b79c4355e: start","job_status":"start","queue_duration_s":1.987458,"scheduling_latency_s":1.987471}1483gitlab # [ 150.084177] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"291feefd8ba19d2b79c4355e","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-291feefd8ba19d2b79c4355e: done: 0.009488 sec","job_status":"done","queue_duration_s":1.987458,"scheduling_latency_s":1.987471,"redis_calls":13,"redis_duration_s":0.0014839999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009488,"completed_at":"2026-09-08T04:25:24.383Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 150.130382] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0872c9f7fab84480b4756d20","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0872c9f7fab84480b4756d20: start","job_status":"start","queue_duration_s":1.997809,"scheduling_latency_s":1.997823}1485gitlab # [ 150.131024] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0872c9f7fab84480b4756d20","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0872c9f7fab84480b4756d20: done: 0.009032 sec","job_status":"done","queue_duration_s":1.997809,"scheduling_latency_s":1.997823,"redis_calls":13,"redis_duration_s":0.001782,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001318,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009032,"completed_at":"2026-09-08T04:25:24.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 150.172243] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68b28ee5a072e9e7fa10e5f8","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68b28ee5a072e9e7fa10e5f8: start","job_status":"start","queue_duration_s":2.007748,"scheduling_latency_s":2.007761}1487gitlab # [ 150.173684] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68b28ee5a072e9e7fa10e5f8","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68b28ee5a072e9e7fa10e5f8: done: 0.009148 sec","job_status":"done","queue_duration_s":2.007748,"scheduling_latency_s":2.007761,"redis_calls":13,"redis_duration_s":0.001712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009148,"completed_at":"2026-09-08T04:25:24.403Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 150.220501] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b546f0415f7baa2c0a644656","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b546f0415f7baa2c0a644656: start","job_status":"start","queue_duration_s":2.017699,"scheduling_latency_s":2.017712}1489gitlab # [ 150.221301] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b546f0415f7baa2c0a644656","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b546f0415f7baa2c0a644656: done: 0.012278 sec","job_status":"done","queue_duration_s":2.017699,"scheduling_latency_s":2.017712,"redis_calls":13,"redis_duration_s":0.0048779999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012278,"completed_at":"2026-09-08T04:25:24.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 150.260797] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6a1a4ea6906f5ea20c908a4","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6a1a4ea6906f5ea20c908a4: start","job_status":"start","queue_duration_s":2.031149,"scheduling_latency_s":2.031163}1491gitlab # [ 150.262647] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6a1a4ea6906f5ea20c908a4","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6a1a4ea6906f5ea20c908a4: done: 0.008854 sec","job_status":"done","queue_duration_s":2.031149,"scheduling_latency_s":2.031163,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008854,"completed_at":"2026-09-08T04:25:24.426Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 150.297676] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5269d997bb68d1fcea125403","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5269d997bb68d1fcea125403: start","job_status":"start","queue_duration_s":2.04081,"scheduling_latency_s":2.040822}1493gitlab # [ 150.337179] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5269d997bb68d1fcea125403","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5269d997bb68d1fcea125403: done: 0.008809 sec","job_status":"done","queue_duration_s":2.04081,"scheduling_latency_s":2.040822,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008809,"completed_at":"2026-09-08T04:25:24.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 150.341954] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc83b405d83e3e555edb12b2","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc83b405d83e3e555edb12b2: start","job_status":"start","queue_duration_s":2.050435,"scheduling_latency_s":2.050448}1495gitlab # [ 150.388184] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc83b405d83e3e555edb12b2","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc83b405d83e3e555edb12b2: done: 0.008744 sec","job_status":"done","queue_duration_s":2.050435,"scheduling_latency_s":2.050448,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000455,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008744,"completed_at":"2026-09-08T04:25:24.446Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 150.389161] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68679c0cd5398c11e6ce1b6b","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68679c0cd5398c11e6ce1b6b: start","job_status":"start","queue_duration_s":2.059987,"scheduling_latency_s":2.06}1497gitlab # [ 150.397586] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68679c0cd5398c11e6ce1b6b","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68679c0cd5398c11e6ce1b6b: done: 0.018934 sec","job_status":"done","queue_duration_s":2.059987,"scheduling_latency_s":2.06,"redis_calls":13,"redis_duration_s":0.011215,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018934,"completed_at":"2026-09-08T04:25:24.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 150.436634] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efa0c699ecc4dd417ba4b485","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efa0c699ecc4dd417ba4b485: start","job_status":"start","queue_duration_s":2.080852,"scheduling_latency_s":2.080867}1499gitlab # [ 150.476201] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efa0c699ecc4dd417ba4b485","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efa0c699ecc4dd417ba4b485: done: 0.008976 sec","job_status":"done","queue_duration_s":2.080852,"scheduling_latency_s":2.080867,"redis_calls":13,"redis_duration_s":0.0014960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008976,"completed_at":"2026-09-08T04:25:24.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 150.477179] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1556d9faf289caac11aaaa6b","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1556d9faf289caac11aaaa6b: start","job_status":"start","queue_duration_s":2.090638,"scheduling_latency_s":2.09065}1501gitlab # [ 150.486673] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1556d9faf289caac11aaaa6b","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1556d9faf289caac11aaaa6b: done: 0.008754 sec","job_status":"done","queue_duration_s":2.090638,"scheduling_latency_s":2.09065,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008754,"completed_at":"2026-09-08T04:25:24.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 150.526553] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b47f122bdda3248a51a234fc","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b47f122bdda3248a51a234fc: start","job_status":"start","queue_duration_s":2.10084,"scheduling_latency_s":2.100855}1503gitlab # [ 150.531199] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b47f122bdda3248a51a234fc","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b47f122bdda3248a51a234fc: done: 0.008897 sec","job_status":"done","queue_duration_s":2.10084,"scheduling_latency_s":2.100855,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008897,"completed_at":"2026-09-08T04:25:24.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 150.579321] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f503061df11aea913aeb864","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f503061df11aea913aeb864: start","job_status":"start","queue_duration_s":2.110552,"scheduling_latency_s":2.110565}1505gitlab # [ 150.579933] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:25:24.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f503061df11aea913aeb864","created_at":"2026-09-08T04:25:22.312Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-f6443e4d9b954243","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0feaeeea0caf4cbff3369348c0d8ca56","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:25:22.386Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f503061df11aea913aeb864: done: 0.008813 sec","job_status":"done","queue_duration_s":2.110552,"scheduling_latency_s":2.110565,"redis_calls":13,"redis_duration_s":0.001514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008813,"completed_at":"2026-09-08T04:25:24.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 150.794280] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2375646108 wd_nsec: 23756459311507gitlab # [ 161.826142] setup-gitlab-start[2235]: W, [2026-09-08T04:25:37.085425 #2235] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1508gitlab # [ 183.320307] postgres[2439]: [2439] ERROR: relation "table_without_models" does not exist1509gitlab # [ 183.321716] postgres[2439]: [2439] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1510gitlab # [ 186.389331] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cd21754cb95fba552b177f07","created_at":"2026-09-08T04:26:01.589Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4806cab54dbf8e01257a51b986b54ad","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.644Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cd21754cb95fba552b177f07: start","job_status":"start","queue_duration_s":0.00328,"scheduling_latency_s":0.003334}1511gitlab # [ 186.578541] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cd21754cb95fba552b177f07","created_at":"2026-09-08T04:26:01.589Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4806cab54dbf8e01257a51b986b54ad","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.644Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cd21754cb95fba552b177f07: done: 0.190468 sec","job_status":"done","queue_duration_s":0.00328,"scheduling_latency_s":0.003334,"redis_calls":10,"redis_duration_s":0.032485,"redis_read_bytes":135,"redis_write_bytes":1136,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001244,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.028053,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.028732144,"exclusive_lock_hold_duration_s":0.058842116,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.190468,"completed_at":"2026-09-08T04:26:01.837Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.083197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 186.580512] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a207098246500bb889f2a7bb","created_at":"2026-09-08T04:26:01.696Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cd8e834c397e5b2347f26f48be2d189","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T04:26:01.702Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a207098246500bb889f2a7bb: start","job_status":"start","queue_duration_s":0.138324,"scheduling_latency_s":0.138353}1513gitlab # [ 186.631458] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a207098246500bb889f2a7bb","created_at":"2026-09-08T04:26:01.696Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cd8e834c397e5b2347f26f48be2d189","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T04:26:01.702Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a207098246500bb889f2a7bb: done: 0.01972 sec","job_status":"done","queue_duration_s":0.138324,"scheduling_latency_s":0.138353,"redis_calls":9,"redis_duration_s":0.001426,"redis_read_bytes":7,"redis_write_bytes":1018,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":277,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01972,"completed_at":"2026-09-08T04:26:01.860Z","load_balancing_strategy":"primary","db_duration_s":0.003605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 186.633298] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2539095d1707145ac952c52c","created_at":"2026-09-08T04:26:01.745Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1fc58449389a07a601c13f565a5b6a8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.753Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2539095d1707145ac952c52c: start","job_status":"start","queue_duration_s":0.107156,"scheduling_latency_s":0.10717}1515gitlab # [ 186.685474] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.865Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1516gitlab # [ 186.688917] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2539095d1707145ac952c52c","created_at":"2026-09-08T04:26:01.745Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1fc58449389a07a601c13f565a5b6a8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.753Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2539095d1707145ac952c52c: done: 0.007567 sec","job_status":"done","queue_duration_s":0.107156,"scheduling_latency_s":0.10717,"redis_calls":7,"redis_duration_s":0.000911,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007567,"completed_at":"2026-09-08T04:26:01.868Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 186.724720] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.869Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"08493f23a4363339a985d25e","created_at":"2026-09-08T04:26:01.766Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ca48f78fd4ff39fc68e03fef925a20b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.772Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-08493f23a4363339a985d25e: start","job_status":"start","queue_duration_s":0.097187,"scheduling_latency_s":0.0972}1518gitlab # [ 186.725143] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"08493f23a4363339a985d25e","created_at":"2026-09-08T04:26:01.766Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ca48f78fd4ff39fc68e03fef925a20b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.772Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-08493f23a4363339a985d25e: done: 0.013091 sec","job_status":"done","queue_duration_s":0.097187,"scheduling_latency_s":0.0972,"redis_calls":10,"redis_duration_s":0.001318,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003777016,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.013091,"completed_at":"2026-09-08T04:26:01.882Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002467,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1519gitlab # [ 186.770541] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5252d4f78e206d88a08f18d0","created_at":"2026-09-08T04:26:01.795Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf4fd456fc7d792a6201e20760784a9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.797Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5252d4f78e206d88a08f18d0: start","job_status":"start","queue_duration_s":0.086239,"scheduling_latency_s":0.086252}1520gitlab # [ 186.771225] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.887Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1521gitlab # [ 186.771674] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.890Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5252d4f78e206d88a08f18d0","created_at":"2026-09-08T04:26:01.795Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf4fd456fc7d792a6201e20760784a9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.797Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5252d4f78e206d88a08f18d0: done: 0.007489 sec","job_status":"done","queue_duration_s":0.086239,"scheduling_latency_s":0.086252,"redis_calls":7,"redis_duration_s":0.000973,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007489,"completed_at":"2026-09-08T04:26:01.890Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 186.786670] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"10d67ccbde0bed806edc594d","created_at":"2026-09-08T04:26:01.802Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.806Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-10d67ccbde0bed806edc594d: start","job_status":"start","queue_duration_s":0.084976,"scheduling_latency_s":0.084989}1523gitlab # [ 186.826388] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"10d67ccbde0bed806edc594d","created_at":"2026-09-08T04:26:01.802Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.806Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-10d67ccbde0bed806edc594d: done: 0.015698 sec","job_status":"done","queue_duration_s":0.084976,"scheduling_latency_s":0.084989,"redis_calls":14,"redis_duration_s":0.001831,"redis_read_bytes":15,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000199,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000391,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015698,"completed_at":"2026-09-08T04:26:01.907Z","load_balancing_strategy":"primary","db_duration_s":0.001511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 186.834576] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:01.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8d62a7fa1c5ebc4e2cc5bb94","created_at":"2026-09-08T04:26:01.831Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d33d2c1667ff160d8fb77e23912e9549","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.834Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8d62a7fa1c5ebc4e2cc5bb94: start","job_status":"start","queue_duration_s":0.073444,"scheduling_latency_s":0.073458}1525gitlab # [ 186.848575] runuser[2234]: pam_unix(runuser:session): session closed for user gitlab1526gitlab # [ 186.849487] systemd[1]: Stopping gitlab-sidekiq.service...1527gitlab # [ 186.850188] setup-gitlab-start[2209]: + systemctl restart gitlab-sidekiq.service1528gitlab # [ 186.903904] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.163Z","message":"Shutting down"}1529gitlab # [ 186.924696] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8d62a7fa1c5ebc4e2cc5bb94","created_at":"2026-09-08T04:26:01.831Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d33d2c1667ff160d8fb77e23912e9549","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.834Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8d62a7fa1c5ebc4e2cc5bb94: done: 0.275775 sec","job_status":"done","queue_duration_s":0.073444,"scheduling_latency_s":0.073458,"redis_calls":9,"redis_duration_s":0.003588,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.215,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.215,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000317638,"exclusive_lock_hold_duration_s":0.259499182,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.275775,"completed_at":"2026-09-08T04:26:02.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.250487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 186.966644] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.226Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b39023f476eb89b460d70894","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b39023f476eb89b460d70894: start","job_status":"start","queue_duration_s":0.321672,"scheduling_latency_s":0.321723}1531gitlab # [ 187.020143] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.280Z","message":"No existing merge request to be cleaned up."}1532gitlab # [ 187.049256] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.308Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b39023f476eb89b460d70894","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b39023f476eb89b460d70894: done: 0.08277 sec","job_status":"done","queue_duration_s":0.321672,"scheduling_latency_s":0.321723,"redis_calls":10,"redis_duration_s":0.017534,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.012687,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08277,"completed_at":"2026-09-08T04:26:02.308Z","load_balancing_strategy":"primary","db_duration_s":0.017236,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 187.109096] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.368Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"258e127d1013850dedf735c9","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-258e127d1013850dedf735c9: start","job_status":"start","queue_duration_s":0.464351,"scheduling_latency_s":0.464383}1534gitlab # [ 187.159331] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.419Z","message":"No existing merge request to be cleaned up."}1535gitlab # [ 187.183464] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.442Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"258e127d1013850dedf735c9","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-258e127d1013850dedf735c9: done: 0.074277 sec","job_status":"done","queue_duration_s":0.464351,"scheduling_latency_s":0.464383,"redis_calls":10,"redis_duration_s":0.016251,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003404,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.012847,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074277,"completed_at":"2026-09-08T04:26:02.442Z","load_balancing_strategy":"primary","db_duration_s":0.009402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 187.242823] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.502Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6d9f821d952623bc18a34ca6","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6d9f821d952623bc18a34ca6: start","job_status":"start","queue_duration_s":0.597818,"scheduling_latency_s":0.597881}1537gitlab # [ 187.286467] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.546Z","message":"No existing merge request to be cleaned up."}1538gitlab # [ 187.288969] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.549Z","message":"Scheduler exiting..."}1539gitlab # [ 187.292562] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.552Z","message":"Terminating quiet threads for default capsule"}1540gitlab # [ 187.296562] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.556Z","message":"Scheduler exiting..."}1541gitlab # [ 187.309417] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:02.568Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6d9f821d952623bc18a34ca6","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":1648,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6d9f821d952623bc18a34ca6: done: 0.066805 sec","job_status":"done","queue_duration_s":0.597818,"scheduling_latency_s":0.597881,"redis_calls":10,"redis_duration_s":0.016847,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005353,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066805,"completed_at":"2026-09-08T04:26:02.568Z","load_balancing_strategy":"primary","db_duration_s":0.005841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542gitlab # [ 187.800909] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:03.060Z","message":"Bye!"}1543gitlab # [ 187.828082] sidekiq-cluster[1648]: {"severity":"INFO","time":"2026-09-08T04:26:03.087Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1648,"worker_id":"sidekiq_0","memwd_rss_bytes":936407040,"retry":0}1544gitlab # [ 189.137516] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1545gitlab # [ 189.138308] systemd[1]: Stopped gitlab-sidekiq.service.1546gitlab # [ 189.140348] systemd[1]: gitlab-sidekiq.service: Consumed 58.012s CPU time over 1min 53.203s wall clock time, 960.1M memory peak, 4.4M read from disk.1547gitlab # [ 189.143112] systemd[1]: Started gitlab-sidekiq.service.1548gitlab # [ 189.146257] setup-gitlab-start[2209]: + mkdir -p /var/lib/secrets1549gitlab # [ 189.151538] setup-gitlab-start[2209]: + echo nixbot-test-token-of-20-chars1550gitlab # [ 189.153080] setup-gitlab-start[2209]: + chmod 644 /var/lib/secrets/gitlab-token1551gitlab # [ 189.158728] setup-gitlab-start[2209]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1552gitlab # [ 189.605807] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:04 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 4061553gitlab # [ 190.165780] gitaly[932]: time="2026-09-08T04:26:05.425Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.424" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1554gitlab # [ 190.186911] gitaly[932]: time="2026-09-08T04:26:05.425Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.424" grpc.time_ms=0.481 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1555gitlab # [ 190.309316] gitaly[932]: time="2026-09-08T04:26:05.569Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.148" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.568" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1556gitlab # [ 190.331678] gitaly[932]: time="2026-09-08T04:26:05.569Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.148" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.568" grpc.time_ms=0.484 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1557gitlab # [ 190.428278] gitaly[932]: time="2026-09-08T04:26:05.688Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN diskcache=d59fc121-a75a-4a72-91c3-a0bae1d570df grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T04:26:15.206" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.627" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1558gitlab # [ 190.450853] gitaly[932]: time="2026-09-08T04:26:05.688Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1577 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T04:26:15.206" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.627" grpc.time_ms=61.167 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1559gitlab # [ 190.451653] gitaly[932]: time="2026-09-08T04:26:05.706Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.285" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.706" pid=932 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1560gitlab # [ 190.453125] gitaly[932]: time="2026-09-08T04:26:05.707Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.285" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.706" grpc.time_ms=0.82 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1561gitlab # [ 190.504466] gitaly[932]: time="2026-09-08T04:26:05.764Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:26:15.046" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.756" grpc.time_ms=7.129 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1562gitlab # [ 190.595931] gitaly[932]: time="2026-09-08T04:26:05.856Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.855" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1563gitlab # [ 190.618392] gitaly[932]: time="2026-09-08T04:26:05.878Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.855" grpc.time_ms=0.403 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1564gitlab # [ 190.670902] gitaly[932]: time="2026-09-08T04:26:05.930Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN diskcache=fa94c92c-a699-4fb1-bdc5-09306fadea33 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T04:26:15.171" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.882" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1565gitlab # [ 190.688631] gitaly[932]: time="2026-09-08T04:26:05.948Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1584 command.oublock=32 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-08T04:26:15.171" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.882" grpc.time_ms=48.66 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1566gitlab # [ 190.698042] gitaly[932]: time="2026-09-08T04:26:05.958Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.246" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.957" pid=932 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1567gitlab # [ 190.699705] gitaly[932]: time="2026-09-08T04:26:05.958Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.246" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.957" grpc.time_ms=0.769 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1568gitlab # [ 190.712606] gitaly[932]: time="2026-09-08T04:26:05.972Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:26:15.253" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.963" grpc.time_ms=8.785 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 190.735913] gitaly[932]: time="2026-09-08T04:26:05.995Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:26:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:05.995" grpc.time_ms=0.198 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1570gitlab # [ 190.746958] gitaly[932]: time="2026-09-08T04:26:06.007Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T04:26:15.015" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:06.006" grpc.time_ms=0.303 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1571gitlab # [ 191.217342] gitaly[932]: time="2026-09-08T04:26:06.476Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM86PSRCCBERX35YRE88TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:26:16.044" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:06.465" grpc.time_ms=10.721 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 191.472925] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:06 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 18411573gitlab # [ 191.474255] setup-gitlab-start[2492]: {"id":1,"message":null,"error":null}1574gitlab # [ 192.216583] sidekiq-cluster[2479]: {"severity":"INFO","time":"2026-09-08T04:26:07.459Z","message":"Starting cluster with 1 processes"}1575gitlab # [ 196.599038] gitaly[932]: time="2026-09-08T04:26:11.858Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DDDDWSVDC2HC3YA81JP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:26:21.139" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:11.850" grpc.time_ms=8.401 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1576gitlab # [ 196.639269] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:11 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1411577gitlab # [ 196.645618] setup-gitlab-start[2209]: + mkdir -p /tmp/test-flake1578gitlab # [ 196.649427] setup-gitlab-start[2209]: + cd /tmp/test-flake1579gitlab # [ 196.649549] setup-gitlab-start[2209]: + git init -b master1580gitlab # [ 196.677693] setup-gitlab-start[2568]: Initialized empty Git repository in /tmp/test-flake/.git/1581gitlab # [ 196.680384] setup-gitlab-start[2209]: + git config user.name test1582gitlab # [ 196.685545] setup-gitlab-start[2209]: + git config user.email test@example.com1583gitlab # [ 196.690372] setup-gitlab-start[2209]: + cat1584gitlab # [ 196.695338] setup-gitlab-start[2209]: + git add flake.nix1585gitlab # [ 196.699521] setup-gitlab-start[2209]: + git commit -m 'initial commit'1586gitlab # [ 196.720870] setup-gitlab-start[2573]: [master (root-commit) c27767c] initial commit1587gitlab # [ 196.721069] setup-gitlab-start[2573]: 1 file changed, 10 insertions(+)1588gitlab # [ 196.721764] setup-gitlab-start[2573]: create mode 100644 flake.nix1589gitlab # [ 196.727075] setup-gitlab-start[2209]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1590gitlab # [ 196.736621] setup-gitlab-start[2209]: + git push -u origin master1591gitlab # [ 196.789355] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:12 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 271592gitlab # [ 196.885254] gitaly[932]: time="2026-09-08T04:26:12.145Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DPQHCACHM6YVV2ZY4S1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:12.132" grpc.time_ms=12.518 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1593gitlab # [ 196.914455] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:12 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1191594gitlab # [ 197.308494] gitaly[932]: time="2026-09-08T04:26:12.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:26:22.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:12.558" grpc.time_ms=10.411 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595gitlab # [ 197.348040] gitaly[932]: time="2026-09-08T04:26:12.608Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-08T04:26:42.096" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-08T04:26:12.596" grpc.time_ms=10.968 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596gitlab # [ 197.368261] gitaly[932]: time="2026-09-08T04:26:12.628Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 duration_ms=136ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/allowed"1597gitlab # [ 197.391698] gitaly[932]: time="2026-09-08T04:26:12.651Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 duration_ms=22ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/pre_receive"1598gitlab # [ 197.392259] gitaly[932]: time="2026-09-08T04:26:12.652Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:12.491" grpc.time_ms=161.174 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1599gitlab # [ 197.445145] gitaly[932]: time="2026-09-08T04:26:12.705Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:12.704" grpc.time_ms=0.521 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1600gitlab # [ 197.491740] gitaly[932]: time="2026-09-08T04:26:12.751Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:12.751" grpc.time_ms=0.347 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1601gitlab # [ 197.527229] gitaly[932]: time="2026-09-08T04:26:12.787Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:12.786" grpc.time_ms=0.412 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1602gitlab # [ 197.603687] gitaly[932]: time="2026-09-08T04:26:12.863Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:26:22.142" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:12.853" grpc.time_ms=9.445 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 197.626242] gitaly[932]: time="2026-09-08T04:26:12.886Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1566 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-08T04:26:22.157" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:12.868" grpc.time_ms=18.145 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 197.649305] gitaly[932]: time="2026-09-08T04:26:12.909Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 duration_ms=94ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/post_receive"1605gitlab # [ 197.652646] gitaly[932]: time="2026-09-08T04:26:12.912Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:12.814" grpc.time_ms=98.29 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1606gitlab # [ 197.673140] gitaly[932]: time="2026-09-08T04:26:12.933Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 diskcache=0f8cf369-e4fb-434b-8743-c5a4293ff61e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:12.432" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1607gitlab # [ 197.673341] gitaly[932]: time="2026-09-08T04:26:12.933Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=202 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=872448 command.minflt=13943 command.oublock=32 command.real_time_ms=487 command.spawn_token_fork_ms=0 command.system_time_ms=126 command.user_time_ms=76 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:12.432" grpc.time_ms=500.538 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1608gitlab # [ 197.694323] gitlab-workhorse[833]: time="2026-09-08T04:26:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521609gitlab # [ 197.694646] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:12 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7661610gitlab # [ 197.699312] setup-gitlab-start[2577]: To http://gitlab/root/test-flake.git1611gitlab # [ 197.700424] setup-gitlab-start[2577]: * [new branch] master -> master1612gitlab # [ 197.702101] setup-gitlab-start[2577]: branch 'master' set up to track 'origin/master'.1613gitlab # [ 197.703699] systemd[1]: Finished setup-gitlab.service.1614gitlab # [ 197.707137] systemd[1]: Starting nixbot CI service...1615gitlab: (finished: waiting for unit setup-gitlab.service, in 198.83 seconds)1616gitlab: waiting for unit nixbot.service1617gitlab # [ 199.110138] nixbot[2663]: {"ts": "2026-09-08T04:26:14.369Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1618gitlab # [ 199.146637] nixbot[2663]: {"ts": "2026-09-08T04:26:14.406Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1619gitlab # [ 199.173567] nixbot[2663]: {"ts": "2026-09-08T04:26:14.433Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1620gitlab # [ 199.179618] nixbot[2663]: {"ts": "2026-09-08T04:26:14.439Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1621gitlab # [ 199.185497] nixbot[2663]: {"ts": "2026-09-08T04:26:14.445Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1622gitlab # [ 199.193799] nixbot[2663]: {"ts": "2026-09-08T04:26:14.453Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1623gitlab # [ 199.198859] nixbot[2663]: {"ts": "2026-09-08T04:26:14.458Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1624gitlab # [ 199.204204] nixbot[2663]: {"ts": "2026-09-08T04:26:14.461Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1625gitlab # [ 199.211293] nixbot[2663]: {"ts": "2026-09-08T04:26:14.471Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1626gitlab # [ 199.218163] nixbot[2663]: {"ts": "2026-09-08T04:26:14.476Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1627gitlab # [ 199.229286] nixbot[2663]: {"ts": "2026-09-08T04:26:14.489Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1628gitlab # [ 199.237499] nixbot[2663]: {"ts": "2026-09-08T04:26:14.497Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1629gitlab # [ 199.242852] nixbot[2663]: {"ts": "2026-09-08T04:26:14.502Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1630gitlab # [ 199.249983] nixbot[2663]: {"ts": "2026-09-08T04:26:14.509Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1631gitlab # [ 199.258542] nixbot[2663]: {"ts": "2026-09-08T04:26:14.518Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1632gitlab # [ 199.264728] nixbot[2663]: {"ts": "2026-09-08T04:26:14.524Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1633gitlab # [ 199.271328] nixbot[2663]: {"ts": "2026-09-08T04:26:14.531Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1634gitlab # [ 199.278466] nixbot[2663]: {"ts": "2026-09-08T04:26:14.538Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1635gitlab # [ 199.289064] nixbot[2663]: {"ts": "2026-09-08T04:26:14.549Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1636gitlab # [ 199.295709] nixbot[2663]: {"ts": "2026-09-08T04:26:14.555Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1637gitlab # [ 199.302354] nixbot[2663]: {"ts": "2026-09-08T04:26:14.562Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1638gitlab # [ 199.310521] nixbot[2663]: {"ts": "2026-09-08T04:26:14.570Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1639gitlab # [ 199.314772] nixbot[2663]: {"ts": "2026-09-08T04:26:14.574Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1640gitlab # [ 199.319112] nixbot[2663]: {"ts": "2026-09-08T04:26:14.579Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1641gitlab # [ 199.323106] nixbot[2663]: {"ts": "2026-09-08T04:26:14.583Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1642gitlab # [ 199.328903] nixbot[2663]: {"ts": "2026-09-08T04:26:14.588Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1643gitlab # [ 199.333737] nixbot[2663]: {"ts": "2026-09-08T04:26:14.593Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1644gitlab # [ 199.338217] nixbot[2663]: {"ts": "2026-09-08T04:26:14.598Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1645gitlab # [ 199.342783] nixbot[2663]: {"ts": "2026-09-08T04:26:14.602Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1646gitlab # [ 199.362210] nixbot[2663]: {"ts": "2026-09-08T04:26:14.622Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1647gitlab # [ 199.368828] nixbot[2663]: {"ts": "2026-09-08T04:26:14.628Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1648gitlab # [ 199.372836] nixbot[2663]: {"ts": "2026-09-08T04:26:14.632Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1649gitlab # [ 199.377459] nixbot[2663]: {"ts": "2026-09-08T04:26:14.637Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1650gitlab # [ 199.383484] nixbot[2663]: {"ts": "2026-09-08T04:26:14.643Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1651gitlab # [ 199.750738] nixbot[2663]: INFO: Started server process [2663]1652gitlab # [ 199.751371] nixbot[2663]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1653gitlab # [ 199.787870] nixbot[2663]: INFO: ::ffff:127.0.0.1:53612 - "GET /health HTTP/1.1" 200 OK1654gitlab # [ 199.795166] systemd[1]: Started nixbot CI service.1655gitlab # [ 199.799535] systemd[1]: Reached target Multi-User System.1656gitlab # [ 199.801991] systemd[1]: Startup finished in 1.176s (kernel) + 3.635s (initrd) + 3min 14.990s (userspace) = 3min 19.801s.1657gitlab: (finished: waiting for unit nixbot.service, in 2.10 seconds)1658gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1659gitlab # [ 200.314817] nixbot[2663]: INFO: ::ffff:127.0.0.1:53624 - "GET /health HTTP/1.1" 200 OK1660gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1661subtest: project discovered and webhook registered1662gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1663gitlab # [ 200.406970] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 511664gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1665gitlab # [ 200.499329] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:15 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 7121666gitlab # [ 200.504854] nixbot[2663]: {"ts": "2026-09-08T04:26:15.760Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1667gitlab # [ 200.523214] nixbot[2663]: {"ts": "2026-09-08T04:26:15.783Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1668gitlab # [ 200.570090] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 361669gitlab # [ 200.570317] nixbot[2663]: {"ts": "2026-09-08T04:26:15.828Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1670gitlab # [ 200.571063] nixbot[2663]: {"ts": "2026-09-08T04:26:15.829Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1671gitlab # [ 200.736737] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1521672gitlab # [ 200.737599] nixbot[2663]: {"ts": "2026-09-08T04:26:15.997Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1673gitlab # [ 200.947693] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1961674gitlab # [ 200.952885] nixbot[2663]: {"ts": "2026-09-08T04:26:16.209Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1675gitlab # [ 201.037455] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 761676gitlab # [ 201.042453] nixbot[2663]: {"ts": "2026-09-08T04:26:16.298Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1677gitlab # [ 201.092510] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 481678gitlab # [ 201.097453] nixbot[2663]: {"ts": "2026-09-08T04:26:16.353Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1679gitlab # [ 201.152086] gitaly[932]: time="2026-09-08T04:26:16.411Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8HX8G6BPD6XFSBFVKNYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-08T04:26:46.099" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:16.400" grpc.time_ms=11.068 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680gitlab # [ 201.184611] gitaly[932]: time="2026-09-08T04:26:16.420Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8HX8G6BPD6XFSBFVKNYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:26:46.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:16.420" grpc.time_ms=0.511 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1681gitlab # [ 201.204910] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 861682gitlab # [ 201.207093] nixbot[2663]: {"ts": "2026-09-08T04:26:16.447Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1683gitlab # [ 201.209448] gitaly[932]: time="2026-09-08T04:26:16.428Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8HX8G6BPD6XFSBFVKNYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-08T04:26:26.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:16.422" grpc.time_ms=5.09 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1684gitlab # [ 201.247045] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 561685gitlab # [ 201.251793] nixbot[2663]: {"ts": "2026-09-08T04:26:16.507Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1686gitlab # [ 201.256573] nixbot[2663]: {"ts": "2026-09-08T04:26:16.512Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1687gitlab # [ 201.332414] gitaly[932]: time="2026-09-08T04:26:16.592Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8J389Y2WWX2HHMDB5K53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.585" grpc.time_ms=6.561 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1688gitlab # [ 201.361683] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691689gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1690gitlab # [ 201.459288] gitaly[932]: time="2026-09-08T04:26:16.718Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8J5KJR78J6JE3ZEDD7A1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.709" pid=932 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1691gitlab # [ 201.460117] gitaly[932]: time="2026-09-08T04:26:16.718Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8J5KJR78J6JE3ZEDD7A1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.709" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1692gitlab # [ 201.461690] gitaly[932]: time="2026-09-08T04:26:16.721Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8J5KJR78J6JE3ZEDD7A1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.709" grpc.time_ms=9.789 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1693gitlab # [ 201.499153] gitlab-workhorse[833]: time="2026-09-08T04:26:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131694gitlab # [ 201.499718] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1301695gitlab # [ 201.534148] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 971696gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds)1697(finished: subtest: project discovered and webhook registered, in 1.22 seconds)1698subtest: push delivers the webhook1699gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1700gitlab # [ 201.574337] gitaly[932]: time="2026-09-08T04:26:16.834Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8J9WATQ0S2SGAM7XQ6J0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.823" pid=932 remote_ip="2001:db8:1::1" request_sha=6f8a837c60c7a695d5dc9878545904051a38c8a2 response_bytes=223 span.kind=server system=grpc1701gitlab # [ 201.574600] gitaly[932]: time="2026-09-08T04:26:16.834Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8J9WATQ0S2SGAM7XQ6J0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.823" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1702gitlab # [ 201.576682] gitaly[932]: time="2026-09-08T04:26:16.834Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8J9WATQ0S2SGAM7XQ6J0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.823" grpc.time_ms=11.348 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1703gitlab # [ 201.577550] gitlab-workhorse[833]: time="2026-09-08T04:26:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231704gitlab # [ 201.578054] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 701705gitlab # [ 201.682460] gitaly[932]: time="2026-09-08T04:26:16.942Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JE5MKMJ0SMVMABZ7NG2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:16.933" grpc.time_ms=8.275 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1706gitlab # [ 201.699976] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 581707gitlab # [ 201.803623] gitaly[932]: time="2026-09-08T04:26:17.063Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JG36RXZE4E4WVH7JWMB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.051" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1708gitlab # [ 201.803878] gitaly[932]: time="2026-09-08T04:26:17.063Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JG36RXZE4E4WVH7JWMB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.051" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1709gitlab # [ 201.805609] gitaly[932]: time="2026-09-08T04:26:17.063Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JG36RXZE4E4WVH7JWMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.051" grpc.time_ms=12.385 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1710gitlab # [ 201.806555] gitlab-workhorse[833]: time="2026-09-08T04:26:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671711gitlab # [ 201.807631] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1031712gitlab # [ 201.951804] gitaly[932]: time="2026-09-08T04:26:17.211Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JPBCVP35Y5B7V1Q7ZSJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.202" grpc.time_ms=9.539 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1713gitlab # [ 201.952793] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 491714gitlab # [ 202.043253] gitaly[932]: time="2026-09-08T04:26:17.303Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JRTD1B0M7B7T18PN2JG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.288" pid=932 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1715gitlab # [ 202.043509] gitaly[932]: time="2026-09-08T04:26:17.303Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JRTD1B0M7B7T18PN2JG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.288" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1716gitlab # [ 202.088997] gitaly[932]: time="2026-09-08T04:26:17.349Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JRTD1B0M7B7T18PN2JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:17.288" grpc.time_ms=60.806 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1717gitlab # [ 202.102250] gitlab-workhorse[833]: time="2026-09-08T04:26:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091718gitlab # [ 202.102740] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1191719gitlab # [ 202.213121] gitaly[932]: time="2026-09-08T04:26:17.472Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:26:47.170" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:17.471" grpc.time_ms=0.936 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720gitlab # [ 202.256097] gitaly[932]: time="2026-09-08T04:26:17.516Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-08T04:26:47.202" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:17.503" grpc.time_ms=12.55 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721gitlab # [ 202.306498] gitaly[932]: time="2026-09-08T04:26:17.566Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:26:27.135" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:17.556" grpc.time_ms=10.211 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1722gitlab # [ 202.335360] gitaly[932]: time="2026-09-08T04:26:17.595Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:26:27.166" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:17.586" grpc.time_ms=8.18 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1723gitlab # [ 202.389258] gitaly[932]: time="2026-09-08T04:26:17.649Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T04:26:27.228" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:17.648" grpc.time_ms=0.364 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1724gitlab # [ 202.402969] gitaly[932]: time="2026-09-08T04:26:17.663Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=617 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T04:26:27.234" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:17.654" grpc.time_ms=8.198 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725gitlab # [ 202.457091] gitaly[932]: time="2026-09-08T04:26:17.717Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:17.716" grpc.time_ms=0.314 pid=932 span.kind=server system=grpc1726gitlab # [ 202.497788] gitaly[932]: time="2026-09-08T04:26:17.757Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:17.757" grpc.time_ms=0.341 pid=932 span.kind=server system=grpc1727gitlab # [ 202.516927] gitaly[932]: time="2026-09-08T04:26:17.777Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC diskcache=58b840b5-8db8-4551-ab4d-c06eb19d6553 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T04:26:27.256" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:17.676" pid=932 ref=refs/keep-around/c27767c876f9d22995def47e76c52f384dc8783a remote_ip="2001:db8:1::1" revision=c27767c876f9d22995def47e76c52f384dc8783a span.kind=server system=grpc username=root1728gitlab # [ 202.517549] gitaly[932]: time="2026-09-08T04:26:17.777Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=75 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5299 command.oublock=8 command.real_time_ms=90 command.spawn_token_fork_ms=0 command.system_time_ms=41 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T04:26:27.256" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:17.676" grpc.time_ms=100.448 pid=932 ref=refs/keep-around/c27767c876f9d22995def47e76c52f384dc8783a remote_ip="2001:db8:1::1" revision=c27767c876f9d22995def47e76c52f384dc8783a span.kind=server system=grpc username=root1729gitlab # [ 202.521219] gitaly[932]: time="2026-09-08T04:26:17.781Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:26:47.480" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:17.780" grpc.time_ms=0.416 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730gitlab # [ 202.638671] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4841731gitlab # [ 202.645299] nixbot[2663]: {"ts": "2026-09-08T04:26:17.900Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}1732gitlab # [ 202.652435] nixbot[2663]: {"ts": "2026-09-08T04:26:17.912Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-c27767c876f9-0"}1733gitlab # [ 202.812716] nixbot[2663]: {"ts": "2026-09-08T04:26:18.072Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-c27767c876f9-0", "argv0": "bwrap"}1734gitlab # [ 202.907224] systemd[1]: Started Nix Daemon.1735gitlab # [ 202.972466] nix-daemon[2829]: accepted connection from pid 2824, user nixbot1736gitlab # [ 207.403997] nix-daemon[2829]: accepted connection from pid 2871, user nixbot1737gitlab # [ 207.413529] nix-daemon[2829]: accepted connection from pid 2824, user nixbot1738gitlab # [ 212.067758] nix-daemon[2829]: accepted connection from pid 2824, user nixbot1739gitlab # [ 212.160875] nix-daemon[2829]: accepted connection from pid 2921, user nixbot1740gitlab # [ 212.209779] gitaly[932]: time="2026-09-08T04:26:27.469Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8WPGMSMV0NJ7Y2V283NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:26:57.067" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:27.468" grpc.time_ms=0.927 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 212.341802] gitaly[932]: time="2026-09-08T04:26:27.601Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8WPGMSMV0NJ7Y2V283NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:26:37.039" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:27.589" grpc.time_ms=11.193 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 212.428755] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2781743gitlab # [ 212.435324] nixbot[2663]: {"ts": "2026-09-08T04:26:27.688Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}1744gitlab # [ 212.489635] gitaly[932]: time="2026-09-08T04:26:27.749Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8WZBPTB4QP71JDYA85J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:26:57.248" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:27.748" grpc.time_ms=0.871 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 212.517777] gitaly[932]: time="2026-09-08T04:26:27.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8WZBPTB4QP71JDYA85J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:26:37.218" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:27.768" grpc.time_ms=9.051 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 212.571365] nixbot[2663]: {"ts": "2026-09-08T04:26:27.831Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}1747gitlab # [ 212.572209] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1401748gitlab # [ 216.242121] nix-daemon[2829]: accepted connection from pid 2989, user nixbot1749gitlab # [ 216.329246] gitaly[932]: time="2026-09-08T04:26:31.588Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM90Q8SW76RC9X9T8VMHA0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:01.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:31.587" grpc.time_ms=0.84 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1750gitlab # [ 216.370111] gitaly[932]: time="2026-09-08T04:26:31.630Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM90Q8SW76RC9X9T8VMHA0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:26:41.069" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:31.619" grpc.time_ms=10.228 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751gitlab # [ 216.408755] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1391752gitlab # [ 216.410171] nixbot[2663]: {"ts": "2026-09-08T04:26:31.668Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}1753gitlab # [ 216.734471] gitaly[932]: time="2026-09-08T04:26:31.994Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM913T8EK79GCW8ATA980P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:31.987" grpc.time_ms=6.933 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1754gitlab # [ 216.735354] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 641755gitlab # [ 216.774477] gitaly[932]: time="2026-09-08T04:26:32.034Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM915XYPKE07N5N5W55H18 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.028" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1756gitlab # [ 216.775799] gitaly[932]: time="2026-09-08T04:26:32.034Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM915XYPKE07N5N5W55H18 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.028" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1757gitlab # [ 216.776662] gitlab-workhorse[833]: time="2026-09-08T04:26:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671758gitlab # [ 216.796983] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 371759gitlab # [ 216.797631] gitaly[932]: time="2026-09-08T04:26:32.034Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM915XYPKE07N5N5W55H18 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.028" grpc.time_ms=6.004 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1760gitlab # [ 216.860386] gitaly[932]: time="2026-09-08T04:26:32.120Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM918GNMG5DWBZTZ1CD4DN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.114" grpc.time_ms=5.961 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1761gitlab # [ 216.861106] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 391762gitlab # [ 216.902118] gitaly[932]: time="2026-09-08T04:26:32.162Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM919TJPJB329VWPW3DC39 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.156" pid=932 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1763gitlab # [ 216.911454] gitaly[932]: time="2026-09-08T04:26:32.171Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM919TJPJB329VWPW3DC39 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.156" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1764gitlab # [ 216.923577] gitaly[932]: time="2026-09-08T04:26:32.183Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM919TJPJB329VWPW3DC39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:32.156" grpc.time_ms=27.1 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1765gitlab # [ 216.924179] gitlab-workhorse[833]: time="2026-09-08T04:26:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671766gitlab # [ 216.924841] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 611767gitlab # [ 217.346349] sidekiq-cluster[2540]: 2026-09-08T04:26:32.605Z pid=2540 tid=xtw INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2540, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>436330496}1768gitlab # [ 217.517081] sidekiq-cluster[2540]: W, [2026-09-08T04:26:32.776900 #2540] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1769gitlab # [ 217.554321] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:32.814Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1770gitlab # [ 219.228901] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:34.488Z","message":"GitLab reliable fetch activated!"}1771gitlab # [ 219.229712] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:34.488Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1772gitlab # [ 238.378653] postgres[3214]: [3214] ERROR: relation "table_without_models" does not exist1773gitlab # [ 238.379153] postgres[3214]: [3214] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1774gitlab # [ 240.765396] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:56.025Z","message":"Booted Rails 7.2.3.1 application in production environment"}1775gitlab # [ 240.770226] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:56.030Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1776gitlab # [ 240.774868] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:56.030Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1777gitlab # [ 240.778862] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:56.030Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1778gitlab # [ 240.783438] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:56.031Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1779gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 39.41 seconds)1780gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1781gitlab # [ 241.055403] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:56 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 471782gitlab # [ 241.152561] gitaly[932]: time="2026-09-08T04:26:56.412Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9RY0GSSDZE0BNASE41XG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:56.400" grpc.time_ms=11.108 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1783gitlab # [ 241.182669] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:56 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 1211784gitlab # Enumerating objects: 5, done.1785gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1786gitlab # Delta compression using up to 4 threads1787gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1788gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1789gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1790gitlab # [ 241.408730] gitaly[932]: time="2026-09-08T04:26:56.668Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-08T04:27:26.356" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-08T04:26:56.656" grpc.time_ms=11.608 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791gitlab # [ 241.439545] gitaly[932]: time="2026-09-08T04:26:56.681Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z duration_ms=77ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/allowed"1792gitlab # [ 241.451579] gitaly[932]: time="2026-09-08T04:26:56.711Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z duration_ms=29ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/pre_receive"1793gitlab # [ 241.452285] gitaly[932]: time="2026-09-08T04:26:56.712Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:56.603" grpc.time_ms=108.964 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1794gitlab # [ 241.493320] gitaly[932]: time="2026-09-08T04:26:56.753Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:56.752" grpc.time_ms=0.429 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1795gitlab # [ 241.524471] gitaly[932]: time="2026-09-08T04:26:56.784Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:56.784" grpc.time_ms=0.413 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1796gitlab # [ 241.550039] gitaly[932]: time="2026-09-08T04:26:56.810Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:56.809" grpc.time_ms=0.335 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1797gitlab # [ 241.611920] gitaly[932]: time="2026-09-08T04:26:56.872Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z duration_ms=34ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/post_receive"1798gitlab # [ 241.615304] gitaly[932]: time="2026-09-08T04:26:56.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:56.837" grpc.time_ms=37.941 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1799gitlab # [ 241.633705] gitaly[932]: time="2026-09-08T04:26:56.893Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z diskcache=0ee7e57d-81ac-4f9b-8bf5-f7a041350f44 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:56.522" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1800gitlab # To http://gitlab/root/test-flake.git1801gitlab # [ 241.642741] gitaly[932]: time="2026-09-08T04:26:56.894Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=156 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13632 command.oublock=32 command.real_time_ms=362 command.spawn_token_fork_ms=0 command.system_time_ms=93 command.user_time_ms=63 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:26:56.522" grpc.time_ms=371.04 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1802gitlab # c27767c..9b05ceb [ 241.643759] gitlab-workhorse[833]: time="2026-09-08T04:26:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521803gitlab # master -> master[ 241.644513] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:26:56 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 4241804gitlab # 1805gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.72 seconds)1806gitlab: must succeed: git -C /tmp/test-flake rev-parse master1807gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1808gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'1809gitlab # [ 241.726247] nixbot[2663]: INFO: ::ffff:127.0.0.1:35338 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK1810gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.06 seconds)1811gitlab # [ 242.104419] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.363Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c2ca9559c20403df0fe8c163","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c2ca9559c20403df0fe8c163: start","job_status":"start","queue_duration_s":55.458664,"scheduling_latency_s":55.458741}1812gitlab # [ 242.233991] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.493Z","message":"No existing merge request to be cleaned up."}1813gitlab # [ 242.251790] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.511Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c2ca9559c20403df0fe8c163","created_at":"2026-09-08T04:26:01.898Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-0ddde4411cea45f2","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"47e67bd69601005f9c63c2126a155ce8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.904Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c2ca9559c20403df0fe8c163: done: 0.148841 sec","job_status":"done","queue_duration_s":55.458664,"scheduling_latency_s":55.458741,"redis_calls":26,"redis_duration_s":0.005829,"redis_read_bytes":786,"redis_write_bytes":2890,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003533,"redis_feature_flag_read_bytes":778,"redis_feature_flag_write_bytes":1868,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":8,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.148841,"completed_at":"2026-09-08T04:26:57.511Z","load_balancing_strategy":"primary","db_duration_s":0.020972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1814gitlab # [ 242.323151] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0826f2b23b525fadbb8fe7c9","created_at":"2026-09-08T04:26:01.937Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5064d35eccc1c4a7847295a2cb9a9807","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.940Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0826f2b23b525fadbb8fe7c9: start","job_status":"start","queue_duration_s":55.572457,"scheduling_latency_s":55.572474}1815gitlab # [ 242.336634] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0826f2b23b525fadbb8fe7c9","created_at":"2026-09-08T04:26:01.937Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5064d35eccc1c4a7847295a2cb9a9807","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.940Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0826f2b23b525fadbb8fe7c9: done: 0.015063 sec","job_status":"done","queue_duration_s":55.572457,"scheduling_latency_s":55.572474,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":11,"redis_write_bytes":1783,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00075,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1007,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015063,"completed_at":"2026-09-08T04:26:57.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1816gitlab # [ 242.375884] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"00e5bcaf4d854991629d708b","created_at":"2026-09-08T04:26:01.943Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f24d113d46b4c0516ed7413ec2386ce0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T04:26:01.944Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-00e5bcaf4d854991629d708b: start","job_status":"start","queue_duration_s":55.58387,"scheduling_latency_s":55.583883}1817gitlab # [ 242.388571] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.537Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1818gitlab # [ 242.388868] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.541Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"00e5bcaf4d854991629d708b","created_at":"2026-09-08T04:26:01.943Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f24d113d46b4c0516ed7413ec2386ce0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T04:26:01.944Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-00e5bcaf4d854991629d708b: done: 0.012751 sec","job_status":"done","queue_duration_s":55.58387,"scheduling_latency_s":55.583883,"redis_calls":11,"redis_duration_s":0.001345,"redis_read_bytes":9,"redis_write_bytes":1557,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0005,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":756,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012751,"completed_at":"2026-09-08T04:26:57.541Z","load_balancing_strategy":"primary","db_duration_s":0.001134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1819gitlab # [ 242.390040] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"88aed0803340a8b5039f5bca","created_at":"2026-09-08T04:26:01.949Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4fbbe3e3a0a11dc13fe4a44d0141b70","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.950Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-88aed0803340a8b5039f5bca: start","job_status":"start","queue_duration_s":55.591401,"scheduling_latency_s":55.591414}1820gitlab # [ 242.587809] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"88aed0803340a8b5039f5bca","created_at":"2026-09-08T04:26:01.949Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4fbbe3e3a0a11dc13fe4a44d0141b70","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.950Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-88aed0803340a8b5039f5bca: done: 0.305304 sec","job_status":"done","queue_duration_s":55.591401,"scheduling_latency_s":55.591414,"redis_calls":566,"redis_duration_s":0.116936,"redis_read_bytes":564,"redis_write_bytes":49420,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":660,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.115589,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.215992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.305304,"completed_at":"2026-09-08T04:26:57.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1821gitlab # [ 242.589284] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:57.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6c5e75627bff4a996ae7df46","created_at":"2026-09-08T04:26:01.955Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4042eec59387fe09a801ec99321d65e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.957Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6c5e75627bff4a996ae7df46: start","job_status":"start","queue_duration_s":55.891059,"scheduling_latency_s":55.891081}1822gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'1823gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.03 seconds)1824gitlab # [ 242.760155] nixbot[2663]: INFO: ::ffff:127.0.0.1:35348 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK1825gitlab # [ 242.760548] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6c5e75627bff4a996ae7df46","created_at":"2026-09-08T04:26:01.955Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4042eec59387fe09a801ec99321d65e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.957Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6c5e75627bff4a996ae7df46: done: 0.171574 sec","job_status":"done","queue_duration_s":55.891059,"scheduling_latency_s":55.891081,"redis_calls":15,"redis_duration_s":0.0037730000000000003,"redis_read_bytes":13,"redis_write_bytes":1926,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001374,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":939,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001175,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":132,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":132,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":132,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000442515,"exclusive_lock_hold_duration_s":0.15150473,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.171574,"completed_at":"2026-09-08T04:26:58.020Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057787,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1826gitlab # [ 242.830240] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f93278a6d0bca330cdce89dd","created_at":"2026-09-08T04:26:01.961Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12fffa5bcbbd36df805e8cec46be2bb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.962Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f93278a6d0bca330cdce89dd: start","job_status":"start","queue_duration_s":56.127754,"scheduling_latency_s":56.127784}1827gitlab # [ 242.858771] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f93278a6d0bca330cdce89dd","created_at":"2026-09-08T04:26:01.961Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12fffa5bcbbd36df805e8cec46be2bb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.962Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f93278a6d0bca330cdce89dd: done: 0.028578 sec","job_status":"done","queue_duration_s":56.127754,"scheduling_latency_s":56.127784,"redis_calls":11,"redis_duration_s":0.001548,"redis_read_bytes":9,"redis_write_bytes":1607,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000555,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028578,"completed_at":"2026-09-08T04:26:58.118Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1828gitlab # [ 242.896751] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"013c24ce6a7cc230a3a8056d","created_at":"2026-09-08T04:26:01.964Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.965Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-013c24ce6a7cc230a3a8056d: start","job_status":"start","queue_duration_s":56.191507,"scheduling_latency_s":56.19153}1829gitlab # [ 243.010457] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"013c24ce6a7cc230a3a8056d","created_at":"2026-09-08T04:26:01.964Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:01.965Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-013c24ce6a7cc230a3a8056d: done: 0.113296 sec","job_status":"done","queue_duration_s":56.191507,"scheduling_latency_s":56.19153,"redis_calls":72,"redis_duration_s":0.011887,"redis_read_bytes":120,"redis_write_bytes":42965,"redis_feature_flag_calls":12,"redis_feature_flag_duration_s":0.00164,"redis_feature_flag_read_bytes":12,"redis_feature_flag_write_bytes":1856,"redis_queues_calls":2,"redis_queues_duration_s":0.000757,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009019,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.113296,"completed_at":"2026-09-08T04:26:58.269Z","load_balancing_strategy":"primary","db_duration_s":0.002986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1830gitlab # [ 243.080089] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.339Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"010a3877a71f05304dc16221","created_at":"2026-09-08T04:26:01.979Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5acbbe5fba372485c6c17b901c0d430c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.167Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-010a3877a71f05304dc16221: start","job_status":"start","queue_duration_s":56.172353,"scheduling_latency_s":56.172403}1831gitlab # [ 243.120682] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"010a3877a71f05304dc16221","created_at":"2026-09-08T04:26:01.979Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5acbbe5fba372485c6c17b901c0d430c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.167Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-010a3877a71f05304dc16221: done: 0.040294 sec","job_status":"done","queue_duration_s":56.172353,"scheduling_latency_s":56.172403,"redis_calls":13,"redis_duration_s":0.002964,"redis_read_bytes":138,"redis_write_bytes":1285,"redis_cache_calls":2,"redis_cache_duration_s":0.000632,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00065,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":416,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000527721,"duration_s":0.040294,"completed_at":"2026-09-08T04:26:58.379Z","load_balancing_strategy":"primary","db_duration_s":0.00093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1832gitlab # [ 243.158929] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1e708b37adb8c610b9a717ed","created_at":"2026-09-08T04:26:02.175Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8db582b335f8c1e41f7cc9d08aed17b1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.182Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1e708b37adb8c610b9a717ed: start","job_status":"start","queue_duration_s":56.236045,"scheduling_latency_s":56.236085}1833gitlab # [ 243.209657] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1e708b37adb8c610b9a717ed","created_at":"2026-09-08T04:26:02.175Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8db582b335f8c1e41f7cc9d08aed17b1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.182Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1e708b37adb8c610b9a717ed: done: 0.050226 sec","job_status":"done","queue_duration_s":56.236045,"scheduling_latency_s":56.236085,"redis_calls":11,"redis_duration_s":0.0025,"redis_read_bytes":137,"redis_write_bytes":1354,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":470,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001378,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381333,"duration_s":0.050226,"completed_at":"2026-09-08T04:26:58.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1834gitlab # [ 243.248344] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"220937c5507e01f0dffea888","created_at":"2026-09-08T04:26:02.232Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbbac3d864bbdc01d6c61554553400c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T04:26:02.235Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-220937c5507e01f0dffea888: start","job_status":"start","queue_duration_s":56.272541,"scheduling_latency_s":56.272583}1835gitlab # [ 243.278797] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.538Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1836gitlab # [ 243.291261] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"220937c5507e01f0dffea888","created_at":"2026-09-08T04:26:02.232Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbbac3d864bbdc01d6c61554553400c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T04:26:02.235Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-220937c5507e01f0dffea888: done: 0.042859 sec","job_status":"done","queue_duration_s":56.272541,"scheduling_latency_s":56.272583,"redis_calls":11,"redis_duration_s":0.004407,"redis_read_bytes":9,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000963,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002993,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042859,"completed_at":"2026-09-08T04:26:58.550Z","load_balancing_strategy":"primary","db_duration_s":0.001059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1837gitlab # [ 243.328909] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e973bb83fa48c30f6e568908","created_at":"2026-09-08T04:26:02.252Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e77dc41fbe4cf731227be101edd0941f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.259Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e973bb83fa48c30f6e568908: start","job_status":"start","queue_duration_s":56.329186,"scheduling_latency_s":56.32923}1838gitlab # [ 243.392878] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e973bb83fa48c30f6e568908","created_at":"2026-09-08T04:26:02.252Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e77dc41fbe4cf731227be101edd0941f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.259Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e973bb83fa48c30f6e568908: done: 0.063828 sec","job_status":"done","queue_duration_s":56.329186,"scheduling_latency_s":56.32923,"redis_calls":13,"redis_duration_s":0.003714,"redis_read_bytes":11,"redis_write_bytes":1921,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00137,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":752,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000827759,"exclusive_lock_hold_duration_s":0.010753322,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.063828,"completed_at":"2026-09-08T04:26:58.652Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1839gitlab # [ 243.444990] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"845a2880d07b0c79eed9c360","created_at":"2026-09-08T04:26:02.278Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e1f186798a60b88bf9a971bc2b64918","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.290Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-845a2880d07b0c79eed9c360: start","job_status":"start","queue_duration_s":56.414275,"scheduling_latency_s":56.414324}1840gitlab # [ 243.480531] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"845a2880d07b0c79eed9c360","created_at":"2026-09-08T04:26:02.278Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e1f186798a60b88bf9a971bc2b64918","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.290Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-845a2880d07b0c79eed9c360: done: 0.035603 sec","job_status":"done","queue_duration_s":56.414275,"scheduling_latency_s":56.414324,"redis_calls":13,"redis_duration_s":0.00426,"redis_read_bytes":139,"redis_write_bytes":1689,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001085,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":464,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000692546,"exclusive_lock_hold_duration_s":0.005679493,"duration_s":0.035603,"completed_at":"2026-09-08T04:26:58.740Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002299,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1841gitlab # [ 243.482409] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bbc87ba3f565238ee688b9b0","created_at":"2026-09-08T04:26:02.305Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3f296515864c019c9e243bde3a186fa","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.310Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bbc87ba3f565238ee688b9b0: start","job_status":"start","queue_duration_s":56.431855,"scheduling_latency_s":56.431946}1842gitlab # [ 243.522319] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.781Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bbc87ba3f565238ee688b9b0","created_at":"2026-09-08T04:26:02.305Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3f296515864c019c9e243bde3a186fa","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.310Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bbc87ba3f565238ee688b9b0: done: 0.039735 sec","job_status":"done","queue_duration_s":56.431855,"scheduling_latency_s":56.431946,"redis_calls":11,"redis_duration_s":0.003233,"redis_read_bytes":9,"redis_write_bytes":1594,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001195,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.039735,"completed_at":"2026-09-08T04:26:58.781Z","load_balancing_strategy":"primary","db_duration_s":0.011633,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1843gitlab # [ 243.528503] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"59550eefb6c1577f58c9e12d","created_at":"2026-09-08T04:26:02.386Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"474c80ee47a0de26f2c0991147d84863","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.390Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-59550eefb6c1577f58c9e12d: start","job_status":"start","queue_duration_s":56.393348,"scheduling_latency_s":56.393377}1844gitlab # [ 243.568648] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"59550eefb6c1577f58c9e12d","created_at":"2026-09-08T04:26:02.386Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"474c80ee47a0de26f2c0991147d84863","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.390Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-59550eefb6c1577f58c9e12d: done: 0.02105 sec","job_status":"done","queue_duration_s":56.393348,"scheduling_latency_s":56.393377,"redis_calls":11,"redis_duration_s":0.00246,"redis_read_bytes":9,"redis_write_bytes":1341,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000955,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":660,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02105,"completed_at":"2026-09-08T04:26:58.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1845gitlab # [ 243.613394] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.805Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"015f61801a7b9c23f7e74222","created_at":"2026-09-08T04:26:02.407Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78dbacaa14b5532435ab4399f63473d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.416Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-015f61801a7b9c23f7e74222: start","job_status":"start","queue_duration_s":56.389682,"scheduling_latency_s":56.389696}1846gitlab # [ 243.614070] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.815Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1847gitlab # [ 243.614557] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"015f61801a7b9c23f7e74222","created_at":"2026-09-08T04:26:02.407Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78dbacaa14b5532435ab4399f63473d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.416Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-015f61801a7b9c23f7e74222: done: 0.013767 sec","job_status":"done","queue_duration_s":56.389682,"scheduling_latency_s":56.389696,"redis_calls":11,"redis_duration_s":0.002399,"redis_read_bytes":9,"redis_write_bytes":1589,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000836,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013767,"completed_at":"2026-09-08T04:26:58.819Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1848gitlab # [ 243.653765] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9dee7315f4ee000efb715818","created_at":"2026-09-08T04:26:02.432Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccc5a5f87bca016b1c6d78f475e25c7d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.436Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9dee7315f4ee000efb715818: start","job_status":"start","queue_duration_s":56.384026,"scheduling_latency_s":56.38404}1849gitlab # [ 243.654610] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9dee7315f4ee000efb715818","created_at":"2026-09-08T04:26:02.432Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccc5a5f87bca016b1c6d78f475e25c7d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.436Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9dee7315f4ee000efb715818: done: 0.01948 sec","job_status":"done","queue_duration_s":56.384026,"scheduling_latency_s":56.38404,"redis_calls":13,"redis_duration_s":0.002881,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000829,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000412064,"exclusive_lock_hold_duration_s":0.005054553,"duration_s":0.01948,"completed_at":"2026-09-08T04:26:58.840Z","load_balancing_strategy":"primary","db_duration_s":0.003577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1850gitlab # [ 243.696978] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.842Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fb60c2aa09be8e2f44b131cb","created_at":"2026-09-08T04:26:02.496Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f80055f43a79c19453dc9da06973baae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.501Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fb60c2aa09be8e2f44b131cb: start","job_status":"start","queue_duration_s":56.34073,"scheduling_latency_s":56.340744}1851gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'1852gitlab # [ 243.697684] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.866Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fb60c2aa09be8e2f44b131cb","created_at":"2026-09-08T04:26:02.496Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f80055f43a79c19453dc9da06973baae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.501Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fb60c2aa09be8e2f44b131cb: done: 0.024078 sec","job_status":"done","queue_duration_s":56.34073,"scheduling_latency_s":56.340744,"redis_calls":17,"redis_duration_s":0.0035120000000000004,"redis_read_bytes":15,"redis_write_bytes":2228,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002158,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1517,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024078,"completed_at":"2026-09-08T04:26:58.866Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853gitlab # [ 243.759291] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0774b2a898a08cc647a6f5c9","created_at":"2026-09-08T04:26:02.536Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5a1cc9bb7dc5c2dbf7b6c470e967e60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.540Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0774b2a898a08cc647a6f5c9: start","job_status":"start","queue_duration_s":56.327018,"scheduling_latency_s":56.327033}1854gitlab # [ 243.759906] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.876Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1855gitlab # [ 243.760598] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0774b2a898a08cc647a6f5c9","created_at":"2026-09-08T04:26:02.536Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-d7ea8f70d29a4ab7","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5a1cc9bb7dc5c2dbf7b6c470e967e60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:02.540Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0774b2a898a08cc647a6f5c9: done: 0.013889 sec","job_status":"done","queue_duration_s":56.327018,"scheduling_latency_s":56.327033,"redis_calls":11,"redis_duration_s":0.002436,"redis_read_bytes":9,"redis_write_bytes":1598,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000947,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":772,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001057,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013889,"completed_at":"2026-09-08T04:26:58.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1856gitlab # [ 243.782127] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:58.882Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1e8d1ae3477a3add8fb52a37","created_at":"2026-09-08T04:26:05.786Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:05.833Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1e8d1ae3477a3add8fb52a37: start","job_status":"start","queue_duration_s":53.049354,"scheduling_latency_s":53.049368}1857gitlab # [ 243.795689] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.014Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1e8d1ae3477a3add8fb52a37","created_at":"2026-09-08T04:26:05.786Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:05.833Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1e8d1ae3477a3add8fb52a37: done: 0.132267 sec","job_status":"done","queue_duration_s":53.049354,"scheduling_latency_s":53.049368,"redis_calls":13,"redis_duration_s":0.002555,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000597,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.109867316,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.132267,"completed_at":"2026-09-08T04:26:59.014Z","load_balancing_strategy":"primary","db_duration_s":0.076925,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1858gitlab # [ 243.840327] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.016Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"757835ae080fa4a65b1a2e1f","created_at":"2026-09-08T04:26:05.846Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:05.851Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-757835ae080fa4a65b1a2e1f: start","job_status":"start","queue_duration_s":53.164609,"scheduling_latency_s":53.164624}1859gitlab # [ 243.848789] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.092Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"757835ae080fa4a65b1a2e1f","created_at":"2026-09-08T04:26:05.846Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:05.851Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-757835ae080fa4a65b1a2e1f: done: 0.076454 sec","job_status":"done","queue_duration_s":53.164609,"scheduling_latency_s":53.164624,"redis_calls":14,"redis_duration_s":0.00346,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000821,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001507,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.059030408,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.076454,"completed_at":"2026-09-08T04:26:59.092Z","load_balancing_strategy":"primary","db_duration_s":0.022186,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1860gitlab # [ 243.887133] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.093Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"bddcc664824fcdcb4d2b6281","created_at":"2026-09-08T04:26:06.367Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:06.371Z","job_size_bytes":5,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-bddcc664824fcdcb4d2b6281: start","job_status":"start","queue_duration_s":52.721862,"scheduling_latency_s":52.721877}1861gitlab # [ 243.950949] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.210Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"bddcc664824fcdcb4d2b6281","created_at":"2026-09-08T04:26:06.367Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:06.371Z","job_size_bytes":5,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-bddcc664824fcdcb4d2b6281: done: 0.116922 sec","job_status":"done","queue_duration_s":52.721862,"scheduling_latency_s":52.721877,"redis_calls":12,"redis_duration_s":0.002999,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000726,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001813,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384965,"duration_s":0.116922,"completed_at":"2026-09-08T04:26:59.210Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025492,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1862gitlab # [ 243.972467] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.212Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"d0a04a1101044fb0e73e9eeb","created_at":"2026-09-08T04:26:06.394Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:06.397Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-d0a04a1101044fb0e73e9eeb: start","job_status":"start","queue_duration_s":52.814978,"scheduling_latency_s":52.814994}1863gitlab # [ 243.974263] nixbot[2663]: INFO: ::ffff:127.0.0.1:33696 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK1864gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.25 seconds)1865gitlab # [ 244.138502] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.397Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"d0a04a1101044fb0e73e9eeb","created_at":"2026-09-08T04:26:06.394Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:06.397Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-d0a04a1101044fb0e73e9eeb: done: 0.18524 sec","job_status":"done","queue_duration_s":52.814978,"scheduling_latency_s":52.814994,"redis_calls":11,"redis_duration_s":0.003205,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000719,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001805,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000681,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.156,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.156,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000288584,"duration_s":0.18524,"completed_at":"2026-09-08T04:26:59.397Z","load_balancing_strategy":"primary","db_duration_s":0.075285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1866gitlab # [ 244.214523] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.474Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"d83d34e84ed2bd320a6830af","created_at":"2026-09-08T04:26:06.399Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:06.402Z","job_size_bytes":89,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-d83d34e84ed2bd320a6830af: start","job_status":"start","queue_duration_s":53.071561,"scheduling_latency_s":53.071596}1867gitlab # [ 244.279600] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.538Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"d83d34e84ed2bd320a6830af","created_at":"2026-09-08T04:26:06.399Z","trace_propagation_headers":{"sentry-trace":"3dded18935ed4f099e5ae44d5632ab7d-df68d8d2efa74db1","baggage":"sentry-trace_id=3dded18935ed4f099e5ae44d5632ab7d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM86PSRCCBERX35YRE88TN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:06.402Z","job_size_bytes":89,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-d83d34e84ed2bd320a6830af: done: 0.064898 sec","job_status":"done","queue_duration_s":53.071561,"scheduling_latency_s":53.071596,"redis_calls":11,"redis_duration_s":0.002819,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000808,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000521016,"duration_s":0.064898,"completed_at":"2026-09-08T04:26:59.538Z","load_balancing_strategy":"primary","db_duration_s":0.007507,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1868gitlab # [ 244.341868] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.601Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 c27767c876f9d22995def47e76c52f384dc8783a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a1edc92121e90730878187c6","created_at":"2026-09-08T04:26:12.840Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-78bf88a2320f418c","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:12.844Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a1edc92121e90730878187c6: start","job_status":"start","queue_duration_s":46.757029,"scheduling_latency_s":46.757068}1869gitlab # [ 244.362444] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:26:59.622Z","message":"changes: \"0000000000000000000000000000000000000000 c27767c876f9d22995def47e76c52f384dc8783a refs/heads/master\\n\""}1870gitlab # [ 244.415868] gitaly[932]: time="2026-09-08T04:26:59.675Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:09.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:59.663" grpc.time_ms=11.957 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1871gitlab # [ 244.434296] gitaly[932]: time="2026-09-08T04:26:59.691Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:27:09.022" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:59.682" grpc.time_ms=8.293 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1872gitlab # [ 244.481166] gitaly[932]: time="2026-09-08T04:26:59.740Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-08T04:27:09.070" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:59.731" grpc.time_ms=9.011 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 244.525549] gitaly[932]: time="2026-09-08T04:26:59.785Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:59.784" grpc.time_ms=0.582 pid=932 span.kind=server system=grpc1874gitlab # [ 244.555253] gitaly[932]: time="2026-09-08T04:26:59.815Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:26:59.814" grpc.time_ms=0.367 pid=932 span.kind=server system=grpc1875gitlab # [ 244.573858] gitaly[932]: time="2026-09-08T04:26:59.833Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 diskcache=695c294d-ebf6-47e2-a820-1d8535a8879f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T04:27:09.101" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:59.762" pid=932 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1876gitlab # [ 244.574707] gitaly[932]: time="2026-09-08T04:26:59.834Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=57 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4927 command.oublock=8 command.real_time_ms=64 command.spawn_token_fork_ms=0 command.system_time_ms=34 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T04:27:09.101" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:59.762" grpc.time_ms=72.019 pid=932 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1877gitlab # [ 244.594641] gitaly[932]: time="2026-09-08T04:26:59.854Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-08T04:27:09.194" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:59.854" grpc.time_ms=0.154 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 244.609581] gitaly[932]: time="2026-09-08T04:26:59.869Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=936 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-08T04:27:09.198" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:59.858" grpc.time_ms=11.128 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 244.683433] gitaly[932]: time="2026-09-08T04:26:59.943Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-08T04:27:29.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:26:59.935" grpc.time_ms=7.77 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 244.713608] gitaly[932]: time="2026-09-08T04:26:59.973Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T04:27:09.013" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:26:59.973" grpc.time_ms=0.24 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881gitlab # [ 244.726564] gitaly[932]: time="2026-09-08T04:26:59.983Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-08T04:27:09.016" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-08T04:26:59.977" grpc.time_ms=6.407 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882gitlab # [ 244.746910] gitaly[932]: time="2026-09-08T04:27:00.006Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:09.039" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:26:59.999" grpc.time_ms=7.009 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab # [ 244.762488] gitaly[932]: time="2026-09-08T04:27:00.022Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:10.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:00.014" grpc.time_ms=8.468 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 244.796191] gitaly[932]: time="2026-09-08T04:27:00.056Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-08T04:27:30.042" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:00.042" grpc.time_ms=13.619 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'1886gitlab # [ 245.050648] nixbot[2663]: INFO: ::ffff:127.0.0.1:33706 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK1887gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.04 seconds)1888gitlab # [ 245.287912] gitaly[932]: time="2026-09-08T04:27:00.547Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-08T04:27:10.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:00.545" grpc.time_ms=1.895 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 245.343902] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:00.602Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 c27767c876f9d22995def47e76c52f384dc8783a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a1edc92121e90730878187c6","created_at":"2026-09-08T04:26:12.840Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-78bf88a2320f418c","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:12.844Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a1edc92121e90730878187c6: done: 1.001211 sec","job_status":"done","queue_duration_s":46.757029,"scheduling_latency_s":46.757068,"gitaly_calls":13,"gitaly_duration_s":0.221062,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.052185999999999996,"redis_read_bytes":2108,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003729,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.007291,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001977,"redis_queues_read_bytes":15,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003407,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.020468,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015314,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.071,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.269,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.29,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.071,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.639938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000852343,"exclusive_lock_hold_duration_s":0.036482849000000005,"duration_s":1.001211,"completed_at":"2026-09-08T04:27:00.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.112367,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1890gitlab # [ 245.351105] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:00.605Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"43d72e9d6437a9f0cc6b128d","created_at":"2026-09-08T04:26:17.785Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.790Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-43d72e9d6437a9f0cc6b128d: start","job_status":"start","queue_duration_s":42.814969,"scheduling_latency_s":42.815004}1891gitlab # [ 245.434193] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:00.693Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"43d72e9d6437a9f0cc6b128d","created_at":"2026-09-08T04:26:17.785Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.790Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-43d72e9d6437a9f0cc6b128d: done: 0.08892 sec","job_status":"done","queue_duration_s":42.814969,"scheduling_latency_s":42.815004,"redis_calls":16,"redis_duration_s":0.004189,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002099,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361499,"duration_s":0.08892,"completed_at":"2026-09-08T04:27:00.693Z","load_balancing_strategy":"primary","db_duration_s":0.02192,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1892gitlab # [ 245.435706] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:00.695Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c09b48dd75b62f51425be6b7","created_at":"2026-09-08T04:26:17.790Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.793Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c09b48dd75b62f51425be6b7: start","job_status":"start","queue_duration_s":42.901974,"scheduling_latency_s":42.901989}1893gitlab # [ 245.582169] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:00.841Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c09b48dd75b62f51425be6b7","created_at":"2026-09-08T04:26:17.790Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.793Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c09b48dd75b62f51425be6b7: done: 0.146328 sec","job_status":"done","queue_duration_s":42.901974,"scheduling_latency_s":42.901989,"redis_calls":15,"redis_duration_s":0.003988,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000873,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001668,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.061,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040927,"duration_s":0.146328,"completed_at":"2026-09-08T04:27:00.841Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.071792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1894gitlab # [ 245.585258] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:00.843Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d125e4207233f5c11fbcf83a","created_at":"2026-09-08T04:26:17.882Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.886Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d125e4207233f5c11fbcf83a: start","job_status":"start","queue_duration_s":42.957669,"scheduling_latency_s":42.957694}1895gitlab # [ 245.731485] gitaly[932]: time="2026-09-08T04:27:00.990Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:00.990" grpc.time_ms=0.695 pid=932 span.kind=server system=grpc1896gitlab # [ 245.777526] gitaly[932]: time="2026-09-08T04:27:01.037Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:01.036" grpc.time_ms=0.579 pid=932 span.kind=server system=grpc1897gitlab # [ 245.819067] gitaly[932]: time="2026-09-08T04:27:01.079Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:01.078" grpc.time_ms=0.317 pid=932 span.kind=server system=grpc1898gitlab # [ 245.840965] gitaly[932]: time="2026-09-08T04:27:01.101Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC diskcache=c41953cd-51ff-41e9-9841-a4ed7244aa1c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:30.053" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:00.954" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 245.841805] gitaly[932]: time="2026-09-08T04:27:01.101Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=142 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7314 command.oublock=0 command.real_time_ms=137 command.spawn_token_fork_ms=0 command.system_time_ms=97 command.user_time_ms=45 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:30.053" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:00.954" grpc.time_ms=147.073 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 245.998814] gitaly[932]: time="2026-09-08T04:27:01.258Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:31.256" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:01.257" grpc.time_ms=1.415 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'1902gitlab # [ 246.056809] gitaly[932]: time="2026-09-08T04:27:01.316Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8JY5BM86N6RHQT5A5MJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:31.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:01.315" grpc.time_ms=1.042 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 246.080409] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.340Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d125e4207233f5c11fbcf83a","created_at":"2026-09-08T04:26:17.882Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.886Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2540,"job_status":"rescheduled","message":"PipelineProcessWorker JID-d125e4207233f5c11fbcf83a: rescheduled"}1904gitlab # [ 246.091770] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.351Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d125e4207233f5c11fbcf83a","created_at":"2026-09-08T04:26:17.882Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-68d0096dce3b46ee","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:17.886Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d125e4207233f5c11fbcf83a: done: 0.507301 sec","job_status":"done","queue_duration_s":42.957669,"scheduling_latency_s":42.957694,"gitaly_calls":3,"gitaly_duration_s":0.201625,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.033359,"redis_read_bytes":463,"redis_write_bytes":29295,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001121,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00167,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010974,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005244,"redis_queues_read_bytes":40,"redis_queues_write_bytes":15582,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.009292,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000577,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000757,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002603,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.021,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.211503,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.45234743499999996,"duration_s":0.507301,"completed_at":"2026-09-08T04:27:01.351Z","load_balancing_strategy":"primary","db_duration_s":0.073208,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1905gitlab # [ 246.101926] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.354Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","c27767c876f9d22995def47e76c52f384dc8783a 9b05cebdf8aa414871b7f2c72bf1e8b712287066 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a5b40e9a6910b090be130a1b","created_at":"2026-09-08T04:26:56.858Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-59332b35e62243a4","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:56.859Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a5b40e9a6910b090be130a1b: start","job_status":"start","queue_duration_s":4.495039,"scheduling_latency_s":4.495066}1906gitlab # [ 246.155371] nixbot[2663]: INFO: ::ffff:127.0.0.1:33708 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK1907gitlab # [ 246.156062] gitaly[932]: time="2026-09-08T04:27:01.405Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:11.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:01.395" grpc.time_ms=9.098 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 246.156650] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.373Z","message":"changes: \"c27767c876f9d22995def47e76c52f384dc8783a 9b05cebdf8aa414871b7f2c72bf1e8b712287066 refs/heads/master\\n\""}1909gitlab # [ 246.160815] gitaly[932]: time="2026-09-08T04:27:01.420Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-08T04:27:11.019" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:27:01.409" grpc.time_ms=11.034 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 246.164371] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.424Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"df7c7ea6bd2348e64d6c956c","created_at":"2026-09-08T04:27:01.422Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"435d8ab3edf8bfa03cb8be45","job_size_bytes":46,"pid":2540,"job_status":"deduplicated","message":"ProjectCacheWorker JID-df7c7ea6bd2348e64d6c956c: deduplicated: until executing","deduplication.type":"until executing"}1911gitlab # [ 246.179359] gitaly[932]: time="2026-09-08T04:27:01.439Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-08T04:27:11.043" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:01.433" grpc.time_ms=5.609 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 246.191532] gitaly[932]: time="2026-09-08T04:27:01.451Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-08T04:27:31.441" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:01.442" grpc.time_ms=8.97 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 246.205331] gitaly[932]: time="2026-09-08T04:27:01.465Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-08T04:27:11.069" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-08T04:27:01.459" grpc.time_ms=5.987 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 246.215542] gitaly[932]: time="2026-09-08T04:27:01.475Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:11.080" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:01.470" grpc.time_ms=4.632 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.20 seconds)1916gitlab # [ 246.274137] gitaly[932]: time="2026-09-08T04:27:01.534Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-08T04:27:11.143" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:01.533" grpc.time_ms=0.621 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1917gitlab # [ 246.307898] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.567Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","c27767c876f9d22995def47e76c52f384dc8783a 9b05cebdf8aa414871b7f2c72bf1e8b712287066 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a5b40e9a6910b090be130a1b","created_at":"2026-09-08T04:26:56.858Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-59332b35e62243a4","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:56.859Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a5b40e9a6910b090be130a1b: done: 0.213188 sec","job_status":"done","queue_duration_s":4.495039,"scheduling_latency_s":4.495066,"gitaly_calls":7,"gitaly_duration_s":0.070631,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.018474,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.001976,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000967,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002433,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007195,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.005903,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077859,"exclusive_lock_hold_duration_s":0.020101717,"duration_s":0.213188,"completed_at":"2026-09-08T04:27:01.567Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022398,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1918gitlab # [ 246.312702] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be490448820d867af8f6bdae","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be490448820d867af8f6bdae: start","job_status":"start","queue_duration_s":3.312237,"scheduling_latency_s":3.312255}1919gitlab # [ 246.324394] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be490448820d867af8f6bdae","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be490448820d867af8f6bdae: done: 0.015273 sec","job_status":"done","queue_duration_s":3.312237,"scheduling_latency_s":3.312255,"redis_calls":15,"redis_duration_s":0.003068,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000421,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015273,"completed_at":"2026-09-08T04:27:01.584Z","load_balancing_strategy":"primary","db_duration_s":0.000524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1920gitlab # [ 246.374342] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bed64f63d349a2de3a9b4afd","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bed64f63d349a2de3a9b4afd: start","job_status":"start","queue_duration_s":3.328673,"scheduling_latency_s":3.328689}1921gitlab # [ 246.375275] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bed64f63d349a2de3a9b4afd","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bed64f63d349a2de3a9b4afd: done: 0.010289 sec","job_status":"done","queue_duration_s":3.328673,"scheduling_latency_s":3.328689,"redis_calls":13,"redis_duration_s":0.002407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001642,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010289,"completed_at":"2026-09-08T04:27:01.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1922gitlab # [ 246.431918] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d25780cd4edeec60d3ff0b","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d25780cd4edeec60d3ff0b: start","job_status":"start","queue_duration_s":3.340147,"scheduling_latency_s":3.34016}1923gitlab # [ 246.432750] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d25780cd4edeec60d3ff0b","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d25780cd4edeec60d3ff0b: done: 0.010358 sec","job_status":"done","queue_duration_s":3.340147,"scheduling_latency_s":3.34016,"redis_calls":13,"redis_duration_s":0.0025050000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010358,"completed_at":"2026-09-08T04:27:01.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1924gitlab # [ 246.480469] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db6bd1d0a52c232a331bd807","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db6bd1d0a52c232a331bd807: start","job_status":"start","queue_duration_s":3.351643,"scheduling_latency_s":3.351656}1925gitlab # [ 246.480988] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db6bd1d0a52c232a331bd807","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db6bd1d0a52c232a331bd807: done: 0.010714 sec","job_status":"done","queue_duration_s":3.351643,"scheduling_latency_s":3.351656,"redis_calls":13,"redis_duration_s":0.00294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010714,"completed_at":"2026-09-08T04:27:01.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1926gitlab # [ 246.539681] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab41ea3b1fb4acce77e00b73","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab41ea3b1fb4acce77e00b73: start","job_status":"start","queue_duration_s":3.363441,"scheduling_latency_s":3.363454}1927gitlab # [ 246.540572] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab41ea3b1fb4acce77e00b73","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab41ea3b1fb4acce77e00b73: done: 0.010375 sec","job_status":"done","queue_duration_s":3.363441,"scheduling_latency_s":3.363454,"redis_calls":13,"redis_duration_s":0.002586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010375,"completed_at":"2026-09-08T04:27:01.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1928gitlab # [ 246.549451] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5a825c69db5130a0a65dffd","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5a825c69db5130a0a65dffd: start","job_status":"start","queue_duration_s":3.374872,"scheduling_latency_s":3.374884}1929gitlab # [ 246.596326] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5a825c69db5130a0a65dffd","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5a825c69db5130a0a65dffd: done: 0.010244 sec","job_status":"done","queue_duration_s":3.374872,"scheduling_latency_s":3.374884,"redis_calls":13,"redis_duration_s":0.0025250000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010244,"completed_at":"2026-09-08T04:27:01.641Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1930gitlab # [ 246.597490] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0be75bee1a7c995c7fa11102","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0be75bee1a7c995c7fa11102: start","job_status":"start","queue_duration_s":3.386341,"scheduling_latency_s":3.386355}1931gitlab # [ 246.642743] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0be75bee1a7c995c7fa11102","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0be75bee1a7c995c7fa11102: done: 0.010407 sec","job_status":"done","queue_duration_s":3.386341,"scheduling_latency_s":3.386355,"redis_calls":13,"redis_duration_s":0.002607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001804,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010407,"completed_at":"2026-09-08T04:27:01.653Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 246.643894] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a44b1f9ce60535f2bd8043c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a44b1f9ce60535f2bd8043c: start","job_status":"start","queue_duration_s":3.397874,"scheduling_latency_s":3.397887}1933gitlab # [ 246.653560] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a44b1f9ce60535f2bd8043c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a44b1f9ce60535f2bd8043c: done: 0.01023 sec","job_status":"done","queue_duration_s":3.397874,"scheduling_latency_s":3.397887,"redis_calls":13,"redis_duration_s":0.002613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01023,"completed_at":"2026-09-08T04:27:01.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1934gitlab # [ 246.698128] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a787879f6e887069cb24410","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a787879f6e887069cb24410: start","job_status":"start","queue_duration_s":3.409179,"scheduling_latency_s":3.409192}1935gitlab # [ 246.699804] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a787879f6e887069cb24410","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a787879f6e887069cb24410: done: 0.010503 sec","job_status":"done","queue_duration_s":3.409179,"scheduling_latency_s":3.409192,"redis_calls":13,"redis_duration_s":0.0024809999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010503,"completed_at":"2026-09-08T04:27:01.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1936gitlab # [ 246.748742] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa9a22a815cc466299a9a04e","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa9a22a815cc466299a9a04e: start","job_status":"start","queue_duration_s":3.420796,"scheduling_latency_s":3.420809}1937gitlab # [ 246.749632] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa9a22a815cc466299a9a04e","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa9a22a815cc466299a9a04e: done: 0.017352 sec","job_status":"done","queue_duration_s":3.420796,"scheduling_latency_s":3.420809,"redis_calls":13,"redis_duration_s":0.002504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001732,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017352,"completed_at":"2026-09-08T04:27:01.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 246.791513] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54fe750de7032de9f076d0bc","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54fe750de7032de9f076d0bc: start","job_status":"start","queue_duration_s":3.439206,"scheduling_latency_s":3.439221}1939gitlab # [ 246.792129] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54fe750de7032de9f076d0bc","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54fe750de7032de9f076d0bc: done: 0.009986 sec","job_status":"done","queue_duration_s":3.439206,"scheduling_latency_s":3.439221,"redis_calls":13,"redis_duration_s":0.002221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001587,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009986,"completed_at":"2026-09-08T04:27:01.706Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1940gitlab # [ 246.837068] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"897ac6df2c066c88c859372d","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-897ac6df2c066c88c859372d: start","job_status":"start","queue_duration_s":3.450023,"scheduling_latency_s":3.450036}1941gitlab # [ 246.838180] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"897ac6df2c066c88c859372d","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-897ac6df2c066c88c859372d: done: 0.008917 sec","job_status":"done","queue_duration_s":3.450023,"scheduling_latency_s":3.450036,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008917,"completed_at":"2026-09-08T04:27:01.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 246.878163] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d700dc2d29adb046d8e9c12","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d700dc2d29adb046d8e9c12: start","job_status":"start","queue_duration_s":3.459762,"scheduling_latency_s":3.459774}1943gitlab # [ 246.921807] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d700dc2d29adb046d8e9c12","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d700dc2d29adb046d8e9c12: done: 0.009173 sec","job_status":"done","queue_duration_s":3.459762,"scheduling_latency_s":3.459774,"redis_calls":13,"redis_duration_s":0.001555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009173,"completed_at":"2026-09-08T04:27:01.725Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 246.922934] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45c7a3ac2bab00b60156a3eb","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45c7a3ac2bab00b60156a3eb: start","job_status":"start","queue_duration_s":3.469761,"scheduling_latency_s":3.469775}1945gitlab # [ 246.929732] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45c7a3ac2bab00b60156a3eb","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45c7a3ac2bab00b60156a3eb: done: 0.016191 sec","job_status":"done","queue_duration_s":3.469761,"scheduling_latency_s":3.469775,"redis_calls":13,"redis_duration_s":0.0038859999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016191,"completed_at":"2026-09-08T04:27:01.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1946gitlab # [ 246.983750] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe41766381721364301f545","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe41766381721364301f545: start","job_status":"start","queue_duration_s":3.488152,"scheduling_latency_s":3.488188}1947gitlab # [ 246.984596] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbe41766381721364301f545","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbe41766381721364301f545: done: 0.021849 sec","job_status":"done","queue_duration_s":3.488152,"scheduling_latency_s":3.488188,"redis_calls":13,"redis_duration_s":0.002618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021849,"completed_at":"2026-09-08T04:27:01.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1948gitlab # [ 247.039736] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7fc9e6a652fec2a23c47c27","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7fc9e6a652fec2a23c47c27: start","job_status":"start","queue_duration_s":3.5118,"scheduling_latency_s":3.511825}1949gitlab # [ 247.040956] gitaly[932]: time="2026-09-08T04:27:02.193Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-08T10:27:02.187" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:27:02.188" grpc.time_ms=4.915 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950gitlab # [ 247.041644] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7fc9e6a652fec2a23c47c27","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7fc9e6a652fec2a23c47c27: done: 0.017016 sec","job_status":"done","queue_duration_s":3.5118,"scheduling_latency_s":3.511825,"redis_calls":13,"redis_duration_s":0.002313,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017016,"completed_at":"2026-09-08T04:27:01.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951gitlab # [ 247.089987] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9a4a1ca10ff8e21618b4955","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9a4a1ca10ff8e21618b4955: start","job_status":"start","queue_duration_s":3.530267,"scheduling_latency_s":3.530293}1952gitlab # [ 247.090805] gitaly[932]: time="2026-09-08T04:27:02.206Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-08T04:27:32.199" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.199" grpc.time_ms=6.707 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1953gitlab # [ 247.095648] gitaly[932]: time="2026-09-08T04:27:02.276Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 diskcache=caa08616-47ac-4942-93a7-37f701736cf7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-08T10:27:02.248" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.249" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1954gitlab # [ 247.136835] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9a4a1ca10ff8e21618b4955","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9a4a1ca10ff8e21618b4955: done: 0.010834 sec","job_status":"done","queue_duration_s":3.530267,"scheduling_latency_s":3.530293,"redis_calls":13,"redis_duration_s":0.0019160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010834,"completed_at":"2026-09-08T04:27:01.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1955gitlab # [ 247.138541] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6719786a9451aa9281c3031","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6719786a9451aa9281c3031: start","job_status":"start","queue_duration_s":3.542054,"scheduling_latency_s":3.542066}1956gitlab # [ 247.186546] gitaly[932]: time="2026-09-08T04:27:02.276Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1288 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-08T10:27:02.248" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.249" grpc.time_ms=27.177 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 247.187424] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6719786a9451aa9281c3031","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6719786a9451aa9281c3031: done: 0.01026 sec","job_status":"done","queue_duration_s":3.542054,"scheduling_latency_s":3.542066,"redis_calls":13,"redis_duration_s":0.002541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001564,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01026,"completed_at":"2026-09-08T04:27:01.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1958gitlab # [ 247.197432] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78a5bdb8cb8867d3c432c42e","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78a5bdb8cb8867d3c432c42e: start","job_status":"start","queue_duration_s":3.553347,"scheduling_latency_s":3.553361}1959gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'1960gitlab # [ 247.208657] gitaly[932]: time="2026-09-08T04:27:02.333Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-08T04:27:32.325" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.325" grpc.time_ms=8.301 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 247.248838] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78a5bdb8cb8867d3c432c42e","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78a5bdb8cb8867d3c432c42e: done: 0.010451 sec","job_status":"done","queue_duration_s":3.553347,"scheduling_latency_s":3.553361,"redis_calls":13,"redis_duration_s":0.0027679999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010451,"completed_at":"2026-09-08T04:27:01.820Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 247.288673] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"453f76a59a093b38ccce142f","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-453f76a59a093b38ccce142f: start","job_status":"start","queue_duration_s":3.564803,"scheduling_latency_s":3.564816}1963gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.05 seconds)1964gitlab # [ 247.289503] gitaly[932]: time="2026-09-08T04:27:02.359Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:32.358" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.359" grpc.time_ms=0.461 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965gitlab # [ 247.289991] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"453f76a59a093b38ccce142f","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-453f76a59a093b38ccce142f: done: 0.010356 sec","job_status":"done","queue_duration_s":3.564803,"scheduling_latency_s":3.564816,"redis_calls":13,"redis_duration_s":0.00263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010356,"completed_at":"2026-09-08T04:27:01.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1966gitlab # [ 247.330209] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9063c5826fce62e055fb4c1","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9063c5826fce62e055fb4c1: start","job_status":"start","queue_duration_s":3.576212,"scheduling_latency_s":3.576225}1967gitlab # [ 247.330872] gitaly[932]: time="2026-09-08T04:27:02.370Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T04:27:12.000" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:02.361" grpc.time_ms=9.37 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 247.332355] nixbot[2663]: INFO: ::ffff:127.0.0.1:33720 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK1969gitlab # [ 247.333789] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9063c5826fce62e055fb4c1","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9063c5826fce62e055fb4c1: done: 0.010387 sec","job_status":"done","queue_duration_s":3.576212,"scheduling_latency_s":3.576225,"redis_calls":13,"redis_duration_s":0.00259,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010387,"completed_at":"2026-09-08T04:27:01.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1970gitlab # [ 247.387471] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be6a0095be2e7db1a44c1111","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be6a0095be2e7db1a44c1111: start","job_status":"start","queue_duration_s":3.58769,"scheduling_latency_s":3.587703}1971gitlab # [ 247.388326] gitaly[932]: time="2026-09-08T04:27:02.511Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:32.509" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.509" grpc.time_ms=1.422 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 247.390588] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be6a0095be2e7db1a44c1111","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be6a0095be2e7db1a44c1111: done: 0.01021 sec","job_status":"done","queue_duration_s":3.58769,"scheduling_latency_s":3.587703,"redis_calls":13,"redis_duration_s":0.002607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001778,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01021,"completed_at":"2026-09-08T04:27:01.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 247.436775] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18ccddef20f3dfea3b7f5a4b","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18ccddef20f3dfea3b7f5a4b: start","job_status":"start","queue_duration_s":3.598967,"scheduling_latency_s":3.598981}1974gitlab # [ 247.437034] gitaly[932]: time="2026-09-08T04:27:02.518Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-08T04:27:32.514" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:02.514" grpc.time_ms=4.157 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 247.449588] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18ccddef20f3dfea3b7f5a4b","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18ccddef20f3dfea3b7f5a4b: done: 0.01052 sec","job_status":"done","queue_duration_s":3.598967,"scheduling_latency_s":3.598981,"redis_calls":13,"redis_duration_s":0.002653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01052,"completed_at":"2026-09-08T04:27:01.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1976gitlab # [ 247.489703] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c6c4f1ce2221c7db1bccc4c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c6c4f1ce2221c7db1bccc4c: start","job_status":"start","queue_duration_s":3.61057,"scheduling_latency_s":3.610583}1977gitlab # [ 247.494670] gitaly[932]: time="2026-09-08T04:27:02.578Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:12.033" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:02.573" grpc.time_ms=5.202 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978gitlab # [ 247.506250] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c6c4f1ce2221c7db1bccc4c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c6c4f1ce2221c7db1bccc4c: done: 0.009987 sec","job_status":"done","queue_duration_s":3.61057,"scheduling_latency_s":3.610583,"redis_calls":13,"redis_duration_s":0.0024330000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001675,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009987,"completed_at":"2026-09-08T04:27:01.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1979gitlab # [ 247.568904] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"931980bd6ce565c1e8f6be39","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-931980bd6ce565c1e8f6be39: start","job_status":"start","queue_duration_s":3.621604,"scheduling_latency_s":3.621617}1980gitlab # [ 247.569654] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"931980bd6ce565c1e8f6be39","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-931980bd6ce565c1e8f6be39: done: 0.00997 sec","job_status":"done","queue_duration_s":3.621604,"scheduling_latency_s":3.621617,"redis_calls":13,"redis_duration_s":0.00245,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00997,"completed_at":"2026-09-08T04:27:01.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1981gitlab # [ 247.612095] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af4adb8ee1cd229181110cd5","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af4adb8ee1cd229181110cd5: start","job_status":"start","queue_duration_s":3.632645,"scheduling_latency_s":3.632658}1982gitlab # [ 247.612749] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af4adb8ee1cd229181110cd5","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af4adb8ee1cd229181110cd5: done: 0.00999 sec","job_status":"done","queue_duration_s":3.632645,"scheduling_latency_s":3.632658,"redis_calls":13,"redis_duration_s":0.002519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001662,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00999,"completed_at":"2026-09-08T04:27:01.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1983gitlab # [ 247.629923] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d0ab47ac1d1a143da39e5f6","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d0ab47ac1d1a143da39e5f6: start","job_status":"start","queue_duration_s":3.643708,"scheduling_latency_s":3.643721}1984gitlab # [ 247.641054] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d0ab47ac1d1a143da39e5f6","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d0ab47ac1d1a143da39e5f6: done: 0.010215 sec","job_status":"done","queue_duration_s":3.643708,"scheduling_latency_s":3.643721,"redis_calls":13,"redis_duration_s":0.00261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001766,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010215,"completed_at":"2026-09-08T04:27:01.910Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1985gitlab # [ 247.695775] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e98cc151b392fe8b2caf9d55","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e98cc151b392fe8b2caf9d55: start","job_status":"start","queue_duration_s":3.655068,"scheduling_latency_s":3.655082}1986gitlab # [ 247.696592] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e98cc151b392fe8b2caf9d55","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e98cc151b392fe8b2caf9d55: done: 0.010068 sec","job_status":"done","queue_duration_s":3.655068,"scheduling_latency_s":3.655082,"redis_calls":13,"redis_duration_s":0.002483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010068,"completed_at":"2026-09-08T04:27:01.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1987gitlab # [ 247.753784] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a67c133291e3eef42cfd293d","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a67c133291e3eef42cfd293d: start","job_status":"start","queue_duration_s":3.666188,"scheduling_latency_s":3.666202}1988gitlab # [ 247.754725] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a67c133291e3eef42cfd293d","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a67c133291e3eef42cfd293d: done: 0.010262 sec","job_status":"done","queue_duration_s":3.666188,"scheduling_latency_s":3.666202,"redis_calls":13,"redis_duration_s":0.002603,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001821,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010262,"completed_at":"2026-09-08T04:27:01.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1989gitlab # [ 247.795765] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0df979d9245bad60f9c47d06","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0df979d9245bad60f9c47d06: start","job_status":"start","queue_duration_s":3.677549,"scheduling_latency_s":3.677562}1990gitlab # [ 247.796418] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0df979d9245bad60f9c47d06","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0df979d9245bad60f9c47d06: done: 0.010145 sec","job_status":"done","queue_duration_s":3.677549,"scheduling_latency_s":3.677562,"redis_calls":13,"redis_duration_s":0.002582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010145,"completed_at":"2026-09-08T04:27:01.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 247.833632] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73530bbeaf6d500e820b729c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73530bbeaf6d500e820b729c: start","job_status":"start","queue_duration_s":3.688777,"scheduling_latency_s":3.688792}1992gitlab # [ 247.834515] gitaly[932]: time="2026-09-08T04:27:03.032Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-08T04:27:33.022" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.022" grpc.time_ms=9.331 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993gitlab # [ 247.835131] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73530bbeaf6d500e820b729c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73530bbeaf6d500e820b729c: done: 0.01009 sec","job_status":"done","queue_duration_s":3.688777,"scheduling_latency_s":3.688792,"redis_calls":13,"redis_duration_s":0.00247,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001694,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01009,"completed_at":"2026-09-08T04:27:01.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1994gitlab # [ 247.879746] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ab8fb054c8a20878fdb3e39","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ab8fb054c8a20878fdb3e39: start","job_status":"start","queue_duration_s":3.699915,"scheduling_latency_s":3.699928}1995gitlab # [ 247.880414] gitaly[932]: time="2026-09-08T04:27:03.040Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM8DTW2QGFCDXGHV5DEHV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-08T04:27:33.033" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.034" grpc.time_ms=6.89 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996gitlab # [ 247.888957] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ab8fb054c8a20878fdb3e39","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ab8fb054c8a20878fdb3e39: done: 0.010018 sec","job_status":"done","queue_duration_s":3.699915,"scheduling_latency_s":3.699928,"redis_calls":13,"redis_duration_s":0.002458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001687,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010018,"completed_at":"2026-09-08T04:27:01.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1997gitlab # [ 247.924697] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"830dee032b53288d92085fbc","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-830dee032b53288d92085fbc: start","job_status":"start","queue_duration_s":3.711025,"scheduling_latency_s":3.711038}1998gitlab # [ 247.964755] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"830dee032b53288d92085fbc","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-830dee032b53288d92085fbc: done: 0.010009 sec","job_status":"done","queue_duration_s":3.711025,"scheduling_latency_s":3.711038,"redis_calls":13,"redis_duration_s":0.002515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010009,"completed_at":"2026-09-08T04:27:01.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 247.968628] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4238eb04883bc683a24934dd","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4238eb04883bc683a24934dd: start","job_status":"start","queue_duration_s":3.722101,"scheduling_latency_s":3.722114}2000gitlab # [ 248.008730] nixbot[2663]: INFO: ::ffff:127.0.0.1:33728 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2001gitlab # [ 248.009397] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4238eb04883bc683a24934dd","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4238eb04883bc683a24934dd: done: 0.009285 sec","job_status":"done","queue_duration_s":3.722101,"scheduling_latency_s":3.722114,"redis_calls":13,"redis_duration_s":0.001748,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009285,"completed_at":"2026-09-08T04:27:01.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002gitlab # [ 248.047172] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3316c575e5e3588fc9c04d49","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3316c575e5e3588fc9c04d49: start","job_status":"start","queue_duration_s":3.732214,"scheduling_latency_s":3.732227}2003gitlab # [ 248.047895] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3316c575e5e3588fc9c04d49","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3316c575e5e3588fc9c04d49: done: 0.008785 sec","job_status":"done","queue_duration_s":3.732214,"scheduling_latency_s":3.732227,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000455,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008785,"completed_at":"2026-09-08T04:27:01.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 248.092269] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:01.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df639897bf40c5cd35369f05","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df639897bf40c5cd35369f05: start","job_status":"start","queue_duration_s":3.741819,"scheduling_latency_s":3.741833}2005gitlab # [ 248.092870] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df639897bf40c5cd35369f05","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df639897bf40c5cd35369f05: done: 0.008713 sec","job_status":"done","queue_duration_s":3.741819,"scheduling_latency_s":3.741833,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008713,"completed_at":"2026-09-08T04:27:02.007Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006gitlab # [ 248.135874] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa16e869be85d29c82f6d22","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa16e869be85d29c82f6d22: start","job_status":"start","queue_duration_s":3.751343,"scheduling_latency_s":3.751357}2007gitlab # [ 248.136595] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa16e869be85d29c82f6d22","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa16e869be85d29c82f6d22: done: 0.009332 sec","job_status":"done","queue_duration_s":3.751343,"scheduling_latency_s":3.751357,"redis_calls":13,"redis_duration_s":0.0017929999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009332,"completed_at":"2026-09-08T04:27:02.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 248.182643] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66b60346637de0f27fddced5","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66b60346637de0f27fddced5: start","job_status":"start","queue_duration_s":3.761514,"scheduling_latency_s":3.761528}2009gitlab # [ 248.183525] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66b60346637de0f27fddced5","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66b60346637de0f27fddced5: done: 0.008954 sec","job_status":"done","queue_duration_s":3.761514,"scheduling_latency_s":3.761528,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008954,"completed_at":"2026-09-08T04:27:02.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 248.195574] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cfdc22e9f4f57f80fa1ed20","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cfdc22e9f4f57f80fa1ed20: start","job_status":"start","queue_duration_s":3.771263,"scheduling_latency_s":3.771277}2011gitlab # [ 248.235462] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cfdc22e9f4f57f80fa1ed20","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cfdc22e9f4f57f80fa1ed20: done: 0.008832 sec","job_status":"done","queue_duration_s":3.771263,"scheduling_latency_s":3.771277,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008832,"completed_at":"2026-09-08T04:27:02.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2012gitlab # [ 248.241971] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380f89dbc0e0a457ef758d44","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380f89dbc0e0a457ef758d44: start","job_status":"start","queue_duration_s":3.780936,"scheduling_latency_s":3.780948}2013gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'2014gitlab # [ 248.286868] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380f89dbc0e0a457ef758d44","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380f89dbc0e0a457ef758d44: done: 0.008722 sec","job_status":"done","queue_duration_s":3.780936,"scheduling_latency_s":3.780948,"redis_calls":13,"redis_duration_s":0.001452,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008722,"completed_at":"2026-09-08T04:27:02.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015gitlab # [ 248.289169] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29dca980b1e3bc84be02e72c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29dca980b1e3bc84be02e72c: start","job_status":"start","queue_duration_s":3.790474,"scheduling_latency_s":3.790488}2016gitlab # [ 248.300865] gitaly[932]: time="2026-09-08T04:27:03.521Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z diskcache=79ff5e6f-16c8-4ef1-85e3-fd61f02d4b5e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-08T10:27:03.511" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.511" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2017gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.05 seconds)2018gitlab # [ 248.343780] gitaly[932]: time="2026-09-08T04:27:03.521Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=314 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-08T10:27:03.511" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.511" grpc.time_ms=9.943 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2019gitlab # [ 248.344653] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29dca980b1e3bc84be02e72c","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29dca980b1e3bc84be02e72c: done: 0.009931 sec","job_status":"done","queue_duration_s":3.790474,"scheduling_latency_s":3.790488,"redis_calls":13,"redis_duration_s":0.002574,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001697,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009931,"completed_at":"2026-09-08T04:27:02.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 248.356599] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be775f61695c2140e211be9f","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be775f61695c2140e211be9f: start","job_status":"start","queue_duration_s":3.801546,"scheduling_latency_s":3.801559}2021gitlab # [ 248.367800] gitaly[932]: time="2026-09-08T04:27:03.545Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:33.544" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.545" grpc.time_ms=0.502 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2022gitlab # [ 248.408477] nixbot[2663]: INFO: ::ffff:127.0.0.1:33740 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK2023gitlab # [ 248.409516] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be775f61695c2140e211be9f","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be775f61695c2140e211be9f: done: 0.009935 sec","job_status":"done","queue_duration_s":3.801546,"scheduling_latency_s":3.801559,"redis_calls":13,"redis_duration_s":0.002542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001753,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009935,"completed_at":"2026-09-08T04:27:02.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 248.450166] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89d0ca4c56634a5e0620fcac","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89d0ca4c56634a5e0620fcac: start","job_status":"start","queue_duration_s":3.812574,"scheduling_latency_s":3.812588}2025gitlab # [ 248.450867] gitaly[932]: time="2026-09-08T04:27:03.554Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T04:27:13.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:03.548" grpc.time_ms=5.995 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2026gitlab # [ 248.451784] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 382027gitlab # [ 248.451968] gitlab-workhorse[833]: time="2026-09-08T04:27:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672028gitlab # [ 248.452590] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 492029gitlab # [ 248.453152] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89d0ca4c56634a5e0620fcac","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89d0ca4c56634a5e0620fcac: done: 0.010015 sec","job_status":"done","queue_duration_s":3.812574,"scheduling_latency_s":3.812588,"redis_calls":13,"redis_duration_s":0.002547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010015,"completed_at":"2026-09-08T04:27:02.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 248.496848] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d2e5b2a2e5c56f529150c7e","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d2e5b2a2e5c56f529150c7e: start","job_status":"start","queue_duration_s":3.823674,"scheduling_latency_s":3.823688}2031gitlab # [ 248.497660] gitaly[932]: time="2026-09-08T04:27:03.577Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:33.576" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.576" grpc.time_ms=0.498 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032gitlab # [ 248.502242] gitlab-workhorse[833]: time="2026-09-08T04:27:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3382033gitlab # [ 248.504701] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 338 "" "git/2.55.0" 412034gitlab # [ 248.505689] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d2e5b2a2e5c56f529150c7e","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d2e5b2a2e5c56f529150c7e: done: 0.009909 sec","job_status":"done","queue_duration_s":3.823674,"scheduling_latency_s":3.823688,"redis_calls":13,"redis_duration_s":0.00249,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009909,"completed_at":"2026-09-08T04:27:02.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035gitlab # [ 248.562234] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1672ad020b764ce5308bd9af","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1672ad020b764ce5308bd9af: start","job_status":"start","queue_duration_s":3.834651,"scheduling_latency_s":3.834666}2036gitlab # [ 248.562874] gitaly[932]: time="2026-09-08T04:27:03.582Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=75 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-08T04:27:33.580" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.580" grpc.time_ms=1.612 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 248.566621] nixbot[2663]: {"ts": "2026-09-08T04:27:03.795Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2038gitlab # [ 248.568968] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1672ad020b764ce5308bd9af","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1672ad020b764ce5308bd9af: done: 0.010071 sec","job_status":"done","queue_duration_s":3.834651,"scheduling_latency_s":3.834666,"redis_calls":13,"redis_duration_s":0.002631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001681,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010071,"completed_at":"2026-09-08T04:27:02.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab # [ 248.612364] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a640a164d7fddc642a87b487","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a640a164d7fddc642a87b487: start","job_status":"start","queue_duration_s":3.845826,"scheduling_latency_s":3.845839}2040gitlab # [ 248.614423] gitaly[932]: time="2026-09-08T04:27:03.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA01W647JWPTW4BQWG7C0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.645" grpc.time_ms=5.685 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2041gitlab # [ 248.663382] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a640a164d7fddc642a87b487","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a640a164d7fddc642a87b487: done: 0.010115 sec","job_status":"done","queue_duration_s":3.845826,"scheduling_latency_s":3.845839,"redis_calls":13,"redis_duration_s":0.002512,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010115,"completed_at":"2026-09-08T04:27:02.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2042gitlab # [ 248.664027] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c773549c67cb672908a3c865","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c773549c67cb672908a3c865: start","job_status":"start","queue_duration_s":3.857081,"scheduling_latency_s":3.857095}2043gitlab # [ 248.672210] gitaly[932]: time="2026-09-08T04:27:03.674Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=73 command.spawn_token_fork_ms=69 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:13.060" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:03.600" grpc.time_ms=74.365 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044gitlab # [ 248.683845] gitaly[932]: time="2026-09-08T04:27:03.700Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA034HJDGZG3MEWXQAYJ1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.694" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2045gitlab # [ 248.724075] nixbot[2663]: {"ts": "2026-09-08T04:27:03.928Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}2046gitlab # [ 248.724818] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c773549c67cb672908a3c865","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c773549c67cb672908a3c865: done: 0.010509 sec","job_status":"done","queue_duration_s":3.857081,"scheduling_latency_s":3.857095,"redis_calls":13,"redis_duration_s":0.00293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010509,"completed_at":"2026-09-08T04:27:02.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 248.730141] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8de81611a232816cc15faf6d","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8de81611a232816cc15faf6d: start","job_status":"start","queue_duration_s":3.868674,"scheduling_latency_s":3.868687}2048gitlab # [ 248.774977] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1232049gitlab # [ 248.775631] gitaly[932]: time="2026-09-08T04:27:03.700Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA034HJDGZG3MEWXQAYJ1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.694" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2050gitlab # [ 248.776410] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8de81611a232816cc15faf6d","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8de81611a232816cc15faf6d: done: 0.009995 sec","job_status":"done","queue_duration_s":3.868674,"scheduling_latency_s":3.868687,"redis_calls":13,"redis_duration_s":0.002554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009995,"completed_at":"2026-09-08T04:27:02.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051gitlab # [ 248.789274] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"331dc7039215b4b4a4b384ff","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-331dc7039215b4b4a4b384ff: start","job_status":"start","queue_duration_s":3.879829,"scheduling_latency_s":3.879841}2052gitlab # [ 248.801307] gitaly[932]: time="2026-09-08T04:27:03.701Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA034HJDGZG3MEWXQAYJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.694" grpc.time_ms=6.987 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2053gitlab # [ 248.813726] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"331dc7039215b4b4a4b384ff","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-331dc7039215b4b4a4b384ff: done: 0.009876 sec","job_status":"done","queue_duration_s":3.879829,"scheduling_latency_s":3.879841,"redis_calls":13,"redis_duration_s":0.0024089999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009876,"completed_at":"2026-09-08T04:27:02.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 248.859607] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d228128c6efb52fc0624a84","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d228128c6efb52fc0624a84: start","job_status":"start","queue_duration_s":3.890544,"scheduling_latency_s":3.890557}2055gitlab # [ 248.860487] gitaly[932]: time="2026-09-08T04:27:03.744Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA04Q4BRYYHHCYVHKR3M6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.734" pid=932 remote_ip="2001:db8:1::1" request_sha=4f7ac9a864a02a5d1b49042bd6c2a34ce3f82436 response_bytes=338 span.kind=server system=grpc2056gitlab # [ 248.871121] nixbot[2663]: INFO: ::ffff:127.0.0.1:33752 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2057gitlab # [ 248.871851] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d228128c6efb52fc0624a84","created_at":"2026-09-08T04:26:58.183Z","trace_propagation_headers":{"sentry-trace":"7105f7c139e24cabbd2e63172a8247d1-a6812343026a424e","baggage":"sentry-trace_id=7105f7c139e24cabbd2e63172a8247d1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3c8056b83a700461615f2caaae8ec7b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:26:58.256Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d228128c6efb52fc0624a84: done: 0.008825 sec","job_status":"done","queue_duration_s":3.890544,"scheduling_latency_s":3.890557,"redis_calls":13,"redis_duration_s":0.001477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000469,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008825,"completed_at":"2026-09-08T04:27:02.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2058gitlab # [ 248.906252] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.157Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"435d8ab3edf8bfa03cb8be45","created_at":"2026-09-08T04:26:59.694Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:59.701Z","job_size_bytes":46,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-435d8ab3edf8bfa03cb8be45: start","job_status":"start","queue_duration_s":2.456057,"scheduling_latency_s":2.45607}2059gitlab # [ 248.920563] gitaly[932]: time="2026-09-08T04:27:03.744Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1189 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA04Q4BRYYHHCYVHKR3M6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.734" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2060gitlab # [ 248.933122] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.231Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"435d8ab3edf8bfa03cb8be45","created_at":"2026-09-08T04:26:59.694Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:59.701Z","job_size_bytes":46,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-435d8ab3edf8bfa03cb8be45: done: 0.074577 sec","job_status":"done","queue_duration_s":2.456057,"scheduling_latency_s":2.45607,"gitaly_calls":2,"gitaly_duration_s":0.016459,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0061470000000000006,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001184,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000524,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001528,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002429,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000271264,"duration_s":0.074577,"completed_at":"2026-09-08T04:27:02.231Z","load_balancing_strategy":"primary","db_duration_s":0.011712,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2061gitlab # [ 248.981311] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.232Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5cebd0f8a698617c35138cf1","created_at":"2026-09-08T04:26:59.719Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:59.725Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5cebd0f8a698617c35138cf1: start","job_status":"start","queue_duration_s":2.507373,"scheduling_latency_s":2.507388}2062gitlab # [ 248.992635] gitaly[932]: time="2026-09-08T04:27:03.744Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1189 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA04Q4BRYYHHCYVHKR3M6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:03.734" grpc.time_ms=9.626 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2063gitlab # [ 249.048218] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.305Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"5cebd0f8a698617c35138cf1","created_at":"2026-09-08T04:26:59.719Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:59.725Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-5cebd0f8a698617c35138cf1: done: 0.072052 sec","job_status":"done","queue_duration_s":2.507373,"scheduling_latency_s":2.507388,"gitaly_calls":1,"gitaly_duration_s":0.029476,"redis_calls":15,"redis_duration_s":0.003195,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000366,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000585,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05700976,"duration_s":0.072052,"completed_at":"2026-09-08T04:27:02.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2064gitlab # [ 249.090864] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.306Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"ba5c114c2a851501642f538a","created_at":"2026-09-08T04:26:59.849Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:59.851Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ba5c114c2a851501642f538a: start","job_status":"start","queue_duration_s":2.454696,"scheduling_latency_s":2.454713}2065gitlab # [ 249.105153] nixbot[2663]: {"ts": "2026-09-08T04:27:04.201Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}2066gitlab # [ 249.107844] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2682067gitlab # [ 249.110299] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 902068gitlab # [ 249.112390] gitlab-workhorse[833]: time="2026-09-08T04:27:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672069gitlab # [ 249.114327] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 682070gitlab # [ 249.116231] gitaly[932]: time="2026-09-08T04:27:03.846Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:33.545" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.846" grpc.time_ms=0.687 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab # [ 249.128253] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.340Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"ba5c114c2a851501642f538a","created_at":"2026-09-08T04:26:59.849Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-08T04:26:59.851Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ba5c114c2a851501642f538a: done: 0.033838 sec","job_status":"done","queue_duration_s":2.454696,"scheduling_latency_s":2.454713,"gitaly_calls":1,"gitaly_duration_s":0.010165,"redis_calls":16,"redis_duration_s":0.0036710000000000002,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000243,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001652,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000611,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033838,"completed_at":"2026-09-08T04:27:02.340Z","load_balancing_strategy":"primary","db_duration_s":0.002706,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2072gitlab # [ 249.169630] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:02.341Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"93ea701461cd7ca5386aee62","created_at":"2026-09-08T04:27:00.399Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3ab802ac0b2991ad6e224732b3e457cf9af900896ef2be865cf610a3a46658ca","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:00.407Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-93ea701461cd7ca5386aee62: start","job_status":"start","queue_duration_s":1.934143,"scheduling_latency_s":1.93416}2073gitlab # [ 249.169964] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1642074gitlab # [ 249.170648] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 622075gitlab # [ 249.171239] nixbot[2663]: {"ts": "2026-09-08T04:27:04.378Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/c27767c876f9d22995def47e76c52f384dc8783a \"HTTP/1.1 201 Created\""}2076gitlab # [ 249.171731] gitaly[932]: time="2026-09-08T04:27:03.889Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:13.172" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:03.883" grpc.time_ms=6.497 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2077gitlab # [ 249.173112] sidekiq-cluster[2540]: {"severity":"WARN","time":"2026-09-08T04:27:03.061Z","class":"CreatePipelineWorker","correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"c27767c876f9d22995def47e76c52f384dc8783a","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"c27767c876f9d22995def47e76c52f384dc8783a","ref":"refs/heads/master"},"retry":0}2078gitlab # [ 249.173648] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.066Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"93ea701461cd7ca5386aee62","created_at":"2026-09-08T04:27:00.399Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3ab802ac0b2991ad6e224732b3e457cf9af900896ef2be865cf610a3a46658ca","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:00.407Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-93ea701461cd7ca5386aee62: done: 0.724996 sec","job_status":"done","queue_duration_s":1.934143,"scheduling_latency_s":1.93416,"gitaly_calls":7,"gitaly_duration_s":0.050349,"redis_calls":51,"redis_duration_s":0.008357999999999999,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000975,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003754,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001892,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.624615,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199187,"duration_s":0.724996,"completed_at":"2026-09-08T04:27:03.066Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040185,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2079gitlab # [ 249.214723] gitaly[932]: time="2026-09-08T04:27:03.969Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0BVHX1DX2FX0W2WYPS1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:33.468" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:03.968" grpc.time_ms=0.486 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2080gitlab # [ 249.262933] gitlab-workhorse[833]: time="2026-09-08T04:27:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672081gitlab # [ 249.263569] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 542082gitlab # [ 249.264103] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.067Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"31ed847b-b848-4e8e-b371-f074f0ae2ef3\"}"],"class":"WebHookWorker","jid":"bc408e21fce9d473af9b87a8","created_at":"2026-09-08T04:27:00.521Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:baaf129d4490187bfa74fe0a058fe3eb2d1df0c88998a1449e366c044fd9e348","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:00.526Z","job_size_bytes":1751,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bc408e21fce9d473af9b87a8: start","job_status":"start","queue_duration_s":2.540955,"scheduling_latency_s":2.54097}2083gitlab # [ 249.264776] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.169Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"31ed847b-b848-4e8e-b371-f074f0ae2ef3\"}"],"class":"WebHookWorker","jid":"bc408e21fce9d473af9b87a8","created_at":"2026-09-08T04:27:00.521Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-23b0640be3b24db5","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:baaf129d4490187bfa74fe0a058fe3eb2d1df0c88998a1449e366c044fd9e348","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:00.526Z","job_size_bytes":1751,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bc408e21fce9d473af9b87a8: done: 0.101656 sec","job_status":"done","queue_duration_s":2.540955,"scheduling_latency_s":2.54097,"redis_calls":19,"redis_duration_s":0.002468,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000223,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.029783674000015026,"cpu_s":0.062474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222654,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.101656,"completed_at":"2026-09-08T04:27:03.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084gitlab # [ 249.326576] gitaly[932]: time="2026-09-08T04:27:04.056Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-08T04:27:34.041" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.041" grpc.time_ms=14.084 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'2086gitlab # [ 249.327515] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f2057c71eb43c87553f71ec9","created_at":"2026-09-08T04:27:00.934Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:00.944Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f2057c71eb43c87553f71ec9: start","job_status":"start","queue_duration_s":2.227578,"scheduling_latency_s":2.227592}2087gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.04 seconds)2088gitlab # [ 249.328289] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f2057c71eb43c87553f71ec9","created_at":"2026-09-08T04:27:00.934Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:00.944Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f2057c71eb43c87553f71ec9: done: 0.052336 sec","job_status":"done","queue_duration_s":2.227578,"scheduling_latency_s":2.227592,"redis_calls":10,"redis_duration_s":0.001349,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204775,"duration_s":0.052336,"completed_at":"2026-09-08T04:27:03.224Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009123,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2089gitlab # [ 249.367236] gitaly[932]: time="2026-09-08T04:27:04.067Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZM9S2PE79Z10DN1Z7FSM3Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-08T04:27:34.057" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.058" grpc.time_ms=8.889 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090gitlab # [ 249.371607] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.225Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"86e9187d481c3ba2b0b74064","created_at":"2026-09-08T04:27:01.128Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.135Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-86e9187d481c3ba2b0b74064: start","job_status":"start","queue_duration_s":2.090463,"scheduling_latency_s":2.090477}2091gitlab # [ 249.386568] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.251Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"86e9187d481c3ba2b0b74064","created_at":"2026-09-08T04:27:01.128Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.135Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-86e9187d481c3ba2b0b74064: done: 0.025989 sec","job_status":"done","queue_duration_s":2.090463,"scheduling_latency_s":2.090477,"redis_calls":10,"redis_duration_s":0.001508,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239416,"duration_s":0.025989,"completed_at":"2026-09-08T04:27:03.251Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092gitlab # [ 249.451345] gitaly[932]: time="2026-09-08T04:27:04.142Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA0BVHX1DX2FX0W2WYPS1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:14.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:04.135" grpc.time_ms=6.783 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093gitlab # [ 249.452276] nixbot[2663]: INFO: ::ffff:127.0.0.1:33754 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK2094gitlab # [ 249.453196] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.253Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5b22b96fa959da3c63407550","created_at":"2026-09-08T04:27:01.151Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.157Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5b22b96fa959da3c63407550: start","job_status":"start","queue_duration_s":2.095261,"scheduling_latency_s":2.095275}2095gitlab # [ 249.493459] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.274Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5b22b96fa959da3c63407550","created_at":"2026-09-08T04:27:01.151Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.157Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5b22b96fa959da3c63407550: done: 0.021419 sec","job_status":"done","queue_duration_s":2.095261,"scheduling_latency_s":2.095275,"redis_calls":10,"redis_duration_s":0.001298,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205333,"duration_s":0.021419,"completed_at":"2026-09-08T04:27:03.274Z","load_balancing_strategy":"primary","db_duration_s":0.003048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 249.495473] gitaly[932]: time="2026-09-08T04:27:04.217Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA0HZ75N63711383MGSMM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.207" grpc.time_ms=10.012 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2097gitlab # [ 249.539875] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.275Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ea82c1acb8cc72fcebebb18f","created_at":"2026-09-08T04:27:01.174Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.179Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ea82c1acb8cc72fcebebb18f: start","job_status":"start","queue_duration_s":2.095614,"scheduling_latency_s":2.095627}2098gitlab # [ 249.540641] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ea82c1acb8cc72fcebebb18f","created_at":"2026-09-08T04:27:01.174Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.179Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ea82c1acb8cc72fcebebb18f: done: 0.022412 sec","job_status":"done","queue_duration_s":2.095614,"scheduling_latency_s":2.095627,"redis_calls":13,"redis_duration_s":0.001821,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000225,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000365,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000445028,"duration_s":0.022412,"completed_at":"2026-09-08T04:27:03.297Z","load_balancing_strategy":"primary","db_duration_s":0.003128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 249.581258] gitaly[932]: time="2026-09-08T04:27:04.248Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0ME6556T7ZYTXWKP253 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:34.047" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.247" grpc.time_ms=0.588 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab # [ 249.581847] gitaly[932]: time="2026-09-08T04:27:04.287Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0MWYD9EDGW4CRFRD43J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.276" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2101gitlab # [ 249.584579] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.298Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a0995fdde8b76eecf6c0c7d7","created_at":"2026-09-08T04:27:01.180Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.184Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a0995fdde8b76eecf6c0c7d7: start","job_status":"start","queue_duration_s":2.113895,"scheduling_latency_s":2.113908}2102gitlab # [ 249.624613] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a0995fdde8b76eecf6c0c7d7","created_at":"2026-09-08T04:27:01.180Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.184Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a0995fdde8b76eecf6c0c7d7: done: 0.019905 sec","job_status":"done","queue_duration_s":2.113895,"scheduling_latency_s":2.113908,"redis_calls":10,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196673,"duration_s":0.019905,"completed_at":"2026-09-08T04:27:03.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103gitlab # [ 249.640929] gitaly[932]: time="2026-09-08T04:27:04.287Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0MWYD9EDGW4CRFRD43J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.276" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2104gitlab # [ 249.683084] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8c8f8edd826160539397a98f","created_at":"2026-09-08T04:27:01.185Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.190Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8c8f8edd826160539397a98f: start","job_status":"start","queue_duration_s":2.128982,"scheduling_latency_s":2.128996}2105gitlab # [ 249.683830] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.337Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8c8f8edd826160539397a98f","created_at":"2026-09-08T04:27:01.185Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.190Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8c8f8edd826160539397a98f: done: 0.018569 sec","job_status":"done","queue_duration_s":2.128982,"scheduling_latency_s":2.128996,"redis_calls":11,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195276,"duration_s":0.018569,"completed_at":"2026-09-08T04:27:03.337Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 249.722002] gitaly[932]: time="2026-09-08T04:27:04.288Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0MWYD9EDGW4CRFRD43J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.276" grpc.time_ms=11.043 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2107gitlab # [ 249.722677] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.338Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0006701e63ca047aecdadf5e","created_at":"2026-09-08T04:27:01.193Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.197Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0006701e63ca047aecdadf5e: start","job_status":"start","queue_duration_s":2.141688,"scheduling_latency_s":2.141701}2108gitlab # [ 249.733614] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.358Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0006701e63ca047aecdadf5e","created_at":"2026-09-08T04:27:01.193Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.197Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0006701e63ca047aecdadf5e: done: 0.019211 sec","job_status":"done","queue_duration_s":2.141688,"scheduling_latency_s":2.141701,"redis_calls":10,"redis_duration_s":0.001244,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000232432,"duration_s":0.019211,"completed_at":"2026-09-08T04:27:03.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109gitlab # [ 249.780136] gitaly[932]: time="2026-09-08T04:27:04.292Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:34.289" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.290" grpc.time_ms=2.502 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2110gitlab # [ 249.784570] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9566e3a60f436174381defda","created_at":"2026-09-08T04:27:01.197Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.200Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9566e3a60f436174381defda: start","job_status":"start","queue_duration_s":2.158211,"scheduling_latency_s":2.158225}2111gitlab # [ 249.798762] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9566e3a60f436174381defda","created_at":"2026-09-08T04:27:01.197Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.200Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9566e3a60f436174381defda: done: 0.024068 sec","job_status":"done","queue_duration_s":2.158211,"scheduling_latency_s":2.158225,"redis_calls":10,"redis_duration_s":0.001212,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000229,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207847,"duration_s":0.024068,"completed_at":"2026-09-08T04:27:03.383Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2112gitlab # [ 249.839419] gitaly[932]: time="2026-09-08T04:27:04.292Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA0ME6556T7ZYTXWKP253 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:14.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:04.280" grpc.time_ms=11.661 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab # [ 249.849684] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.383Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b0fc7ee100cd9c85cd0cc2a3","created_at":"2026-09-08T04:27:01.207Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.214Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b0fc7ee100cd9c85cd0cc2a3: start","job_status":"start","queue_duration_s":2.169056,"scheduling_latency_s":2.169069}2114gitlab # [ 249.864131] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.406Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b0fc7ee100cd9c85cd0cc2a3","created_at":"2026-09-08T04:27:01.207Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.214Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b0fc7ee100cd9c85cd0cc2a3: done: 0.022272 sec","job_status":"done","queue_duration_s":2.169056,"scheduling_latency_s":2.169069,"redis_calls":10,"redis_duration_s":0.001352,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000216787,"duration_s":0.022272,"completed_at":"2026-09-08T04:27:03.406Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006189,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115gitlab # [ 249.915973] gitaly[932]: time="2026-09-08T04:27:04.326Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:34.324" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.325" grpc.time_ms=1.033 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 249.916671] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.407Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8ec0fcd9cd05f02e8a151d9b","created_at":"2026-09-08T04:27:01.215Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.221Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8ec0fcd9cd05f02e8a151d9b: start","job_status":"start","queue_duration_s":2.186302,"scheduling_latency_s":2.186315}2117gitlab # [ 249.928264] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.421Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8ec0fcd9cd05f02e8a151d9b","created_at":"2026-09-08T04:27:01.215Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.221Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8ec0fcd9cd05f02e8a151d9b: done: 0.014011 sec","job_status":"done","queue_duration_s":2.186302,"scheduling_latency_s":2.186315,"redis_calls":10,"redis_duration_s":0.001256,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203099,"duration_s":0.014011,"completed_at":"2026-09-08T04:27:03.421Z","load_balancing_strategy":"primary","db_duration_s":0.001272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 249.989097] gitaly[932]: time="2026-09-08T04:27:04.381Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA0QZC699EGMH9MH03XGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=200 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.372" grpc.time_ms=8.504 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2119gitlab # [ 249.989802] gitaly[932]: time="2026-09-08T04:27:04.437Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0T0JJF570SF8KVDTRSN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.429" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2120gitlab # [ 249.990459] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"75461dff5119d46804d80118","created_at":"2026-09-08T04:27:01.221Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.226Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-75461dff5119d46804d80118: start","job_status":"start","queue_duration_s":2.195735,"scheduling_latency_s":2.195755}2121gitlab # [ 250.029534] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"75461dff5119d46804d80118","created_at":"2026-09-08T04:27:01.221Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.226Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-75461dff5119d46804d80118: done: 0.021395 sec","job_status":"done","queue_duration_s":2.195735,"scheduling_latency_s":2.195755,"redis_calls":10,"redis_duration_s":0.001521,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000202261,"duration_s":0.021395,"completed_at":"2026-09-08T04:27:03.443Z","load_balancing_strategy":"primary","db_duration_s":0.00372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 250.038654] gitaly[932]: time="2026-09-08T04:27:04.437Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0T0JJF570SF8KVDTRSN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.429" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2123gitlab # [ 250.050619] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.444Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"7a8aa8144fe77537d3700569","created_at":"2026-09-08T04:27:01.234Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.242Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-7a8aa8144fe77537d3700569: start","job_status":"start","queue_duration_s":2.201868,"scheduling_latency_s":2.201882}2124gitlab # [ 250.065640] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.482Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"7a8aa8144fe77537d3700569","created_at":"2026-09-08T04:27:01.234Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.242Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-7a8aa8144fe77537d3700569: done: 0.038317 sec","job_status":"done","queue_duration_s":2.201868,"scheduling_latency_s":2.201882,"redis_calls":14,"redis_duration_s":0.001771,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207568,"duration_s":0.038317,"completed_at":"2026-09-08T04:27:03.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 250.107760] gitaly[932]: time="2026-09-08T04:27:04.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA0T0JJF570SF8KVDTRSN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:04.429" grpc.time_ms=7.89 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2126gitlab # [ 250.116777] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.483Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"654f0925259366cf80c4007b","created_at":"2026-09-08T04:27:01.340Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.343Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-654f0925259366cf80c4007b: start","job_status":"start","queue_duration_s":2.140857,"scheduling_latency_s":2.14087}2127gitlab # [ 250.156788] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.499Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"654f0925259366cf80c4007b","created_at":"2026-09-08T04:27:01.340Z","trace_propagation_headers":{"sentry-trace":"02f0264f861f435c82ac03e0eef457f7-43aa316ce46a4bfa","baggage":"sentry-trace_id=02f0264f861f435c82ac03e0eef457f7,sentry-release=Unknown"},"correlation_id":"01M1ZM8JY5BM86N6RHQT5A5MJC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.343Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-654f0925259366cf80c4007b: done: 0.015598 sec","job_status":"done","queue_duration_s":2.140857,"scheduling_latency_s":2.14087,"redis_calls":8,"redis_duration_s":0.001031,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015598,"completed_at":"2026-09-08T04:27:03.499Z","load_balancing_strategy":"primary","db_duration_s":0.002201,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2128gitlab # [ 250.199743] gitaly[932]: time="2026-09-08T04:27:04.540Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:04.539" grpc.time_ms=0.469 pid=932 span.kind=server system=grpc2129gitlab # [ 250.200544] gitaly[932]: time="2026-09-08T04:27:04.559Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:04.559" grpc.time_ms=0.401 pid=932 span.kind=server system=grpc2130gitlab # [ 250.201209] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.500Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"706292516360324aed218077","created_at":"2026-09-08T04:27:01.428Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"5cebd0f8a698617c35138cf1","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.430Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-706292516360324aed218077: start","job_status":"start","queue_duration_s":2.070024,"scheduling_latency_s":2.070038}2131gitlab # [ 250.201766] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.530Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"706292516360324aed218077","created_at":"2026-09-08T04:27:01.428Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"5cebd0f8a698617c35138cf1","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.430Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-706292516360324aed218077: done: 0.030216 sec","job_status":"done","queue_duration_s":2.070024,"scheduling_latency_s":2.070038,"gitaly_calls":1,"gitaly_duration_s":0.012027,"redis_calls":12,"redis_duration_s":0.001649,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000355,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.019750555,"duration_s":0.030216,"completed_at":"2026-09-08T04:27:03.530Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2132gitlab # [ 250.259083] gitaly[932]: time="2026-09-08T04:27:04.580Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:04.579" grpc.time_ms=0.576 pid=932 span.kind=server system=grpc2133gitlab # [ 250.259641] gitaly[932]: time="2026-09-08T04:27:04.591Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 diskcache=d6af9fe2-4f18-40e0-b166-3b9b200da94c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:34.515" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:04.516" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134gitlab # [ 250.260436] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:03.531Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"9b28527d2039fffad937f57c","created_at":"2026-09-08T04:27:01.506Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:89ba4f7e7ae133d1a2307c54e871a873298f69ed726b84e3d93493ba295db459","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.508Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-9b28527d2039fffad937f57c: start","job_status":"start","queue_duration_s":2.023299,"scheduling_latency_s":2.023314}2135gitlab # [ 250.306535] sidekiq-cluster[2540]: {"severity":"WARN","time":"2026-09-08T04:27:04.079Z","class":"CreatePipelineWorker","correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"c27767c876f9d22995def47e76c52f384dc8783a","after":"9b05cebdf8aa414871b7f2c72bf1e8b712287066","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"9b05cebdf8aa414871b7f2c72bf1e8b712287066","ref":"refs/heads/master"},"retry":0}2136gitlab # [ 250.307180] gitaly[932]: time="2026-09-08T04:27:04.591Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=70 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7367 command.oublock=0 command.real_time_ms=66 command.spawn_token_fork_ms=2 command.system_time_ms=43 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:34.515" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:04.516" grpc.time_ms=74.721 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137gitlab # [ 250.307735] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"9b28527d2039fffad937f57c","created_at":"2026-09-08T04:27:01.506Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:89ba4f7e7ae133d1a2307c54e871a873298f69ed726b84e3d93493ba295db459","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.508Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-9b28527d2039fffad937f57c: done: 0.551136 sec","job_status":"done","queue_duration_s":2.023299,"scheduling_latency_s":2.023314,"gitaly_calls":7,"gitaly_duration_s":0.118504,"redis_calls":33,"redis_duration_s":0.003833,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000402,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001684,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.414905,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.551136,"completed_at":"2026-09-08T04:27:04.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015077,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2138gitlab # [ 250.345700] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.084Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"cdf2bde8-724b-464d-9d65-e523b2c6dea7\"}"],"class":"WebHookWorker","jid":"2998d1c6900b16d3976fbb8a","created_at":"2026-09-08T04:27:01.523Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8eb9e1bdbf6d948e890d95457a6317cc9180656c94996b07c8218dd9683a5ca2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.525Z","job_size_bytes":1749,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2998d1c6900b16d3976fbb8a: start","job_status":"start","queue_duration_s":2.558839,"scheduling_latency_s":2.558856}2139gitlab # [ 250.346873] gitaly[932]: time="2026-09-08T04:27:04.635Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:34.633" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.634" grpc.time_ms=0.898 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2140gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'2141gitlab # [ 250.355781] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.116Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"cdf2bde8-724b-464d-9d65-e523b2c6dea7\"}"],"class":"WebHookWorker","jid":"2998d1c6900b16d3976fbb8a","created_at":"2026-09-08T04:27:01.523Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-5b840c59b0c94005","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8eb9e1bdbf6d948e890d95457a6317cc9180656c94996b07c8218dd9683a5ca2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:01.525Z","job_size_bytes":1749,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2998d1c6900b16d3976fbb8a: done: 0.032663 sec","job_status":"done","queue_duration_s":2.558839,"scheduling_latency_s":2.558856,"redis_calls":14,"redis_duration_s":0.0082,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000226,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007476,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.004926882999995996,"cpu_s":0.01509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.032663,"completed_at":"2026-09-08T04:27:04.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.04 seconds)2143gitlab # [ 250.398307] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.118Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"003f5bf3a263f5b97191233e","created_at":"2026-09-08T04:27:02.219Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-684c6742ec4a4251","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:02.223Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-003f5bf3a263f5b97191233e: start","job_status":"start","queue_duration_s":1.894075,"scheduling_latency_s":1.894091}2144gitlab # [ 250.439838] gitaly[932]: time="2026-09-08T04:27:04.660Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA07W0DY8DE6857KH85G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:34.659" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:04.659" grpc.time_ms=0.697 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145gitlab # [ 250.440455] nixbot[2663]: INFO: ::ffff:127.0.0.1:33764 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK2146gitlab # [ 250.440956] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.155Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"003f5bf3a263f5b97191233e","created_at":"2026-09-08T04:27:02.219Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-684c6742ec4a4251","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:02.223Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-003f5bf3a263f5b97191233e: done: 0.037899 sec","job_status":"done","queue_duration_s":1.894075,"scheduling_latency_s":1.894091,"redis_calls":19,"redis_duration_s":0.0035169999999999997,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001426,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000323,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004826591999999999,"duration_s":0.037899,"completed_at":"2026-09-08T04:27:04.155Z","load_balancing_strategy":"primary","db_duration_s":0.008917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147gitlab # [ 250.463453] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.157Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bc408e21fce9d473af9b87a8"],"class":"WebHooks::LogExecutionWorker","jid":"94a6c33ef10e71d05f04c7ea","created_at":"2026-09-08T04:27:03.158Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-8903fd8f5b794679","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:19929fa33118cc46c0d0c82f809b45446399b30b691441bbb218c155d644efd4","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:03.163Z","job_size_bytes":2593,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-94a6c33ef10e71d05f04c7ea: start","job_status":"start","queue_duration_s":0.993071,"scheduling_latency_s":0.993084}2148gitlab # [ 250.504685] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.193Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bc408e21fce9d473af9b87a8"],"class":"WebHooks::LogExecutionWorker","jid":"94a6c33ef10e71d05f04c7ea","created_at":"2026-09-08T04:27:03.158Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-8903fd8f5b794679","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:19929fa33118cc46c0d0c82f809b45446399b30b691441bbb218c155d644efd4","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:03.163Z","job_size_bytes":2593,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-94a6c33ef10e71d05f04c7ea: done: 0.034707 sec","job_status":"done","queue_duration_s":0.993071,"scheduling_latency_s":0.993084,"redis_calls":10,"redis_duration_s":0.002602,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001572,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000248356,"duration_s":0.034707,"completed_at":"2026-09-08T04:27:04.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149gitlab # [ 250.517849] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.196Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"98832fb6f4469023864ed310","created_at":"2026-09-08T04:27:03.914Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-39a4898619694940","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:03.917Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-98832fb6f4469023864ed310: start","job_status":"start","queue_duration_s":0.279034,"scheduling_latency_s":0.279065}2150gitlab # [ 250.563558] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.332Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"98832fb6f4469023864ed310","created_at":"2026-09-08T04:27:03.914Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-39a4898619694940","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:03.917Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2540,"job_status":"rescheduled","message":"PipelineProcessWorker JID-98832fb6f4469023864ed310: rescheduled"}2151gitlab # [ 250.564395] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.338Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"98832fb6f4469023864ed310","created_at":"2026-09-08T04:27:03.914Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-39a4898619694940","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:03.917Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-98832fb6f4469023864ed310: done: 0.141797 sec","job_status":"done","queue_duration_s":0.279034,"scheduling_latency_s":0.279065,"gitaly_calls":2,"gitaly_duration_s":0.010782,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.01205,"redis_read_bytes":59,"redis_write_bytes":6217,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002519,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000415,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_calls":4,"redis_queues_duration_s":0.002106,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2220,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002015,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":979,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000675,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000166,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070202,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.111905005,"duration_s":0.141797,"completed_at":"2026-09-08T04:27:04.338Z","load_balancing_strategy":"primary","db_duration_s":0.038442,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2152gitlab # [ 250.611072] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.340Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2998d1c6900b16d3976fbb8a"],"class":"WebHooks::LogExecutionWorker","jid":"d0417596062fc006193b4a78","created_at":"2026-09-08T04:27:04.104Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-911fde9754ed425f","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cf765c0904fabb04313108ff3bfabebdee87bcee3289836ccdce47c37ea18a80","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.113Z","job_size_bytes":2592,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-d0417596062fc006193b4a78: start","job_status":"start","queue_duration_s":0.227833,"scheduling_latency_s":0.22785}2153gitlab # [ 250.611656] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.370Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2998d1c6900b16d3976fbb8a"],"class":"WebHooks::LogExecutionWorker","jid":"d0417596062fc006193b4a78","created_at":"2026-09-08T04:27:04.104Z","trace_propagation_headers":{"sentry-trace":"fce3ce41bd8c4e6dac23a482653447d2-911fde9754ed425f","baggage":"sentry-trace_id=fce3ce41bd8c4e6dac23a482653447d2,sentry-release=Unknown"},"correlation_id":"01M1ZM9S2PE79Z10DN1Z7FSM3Z","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cf765c0904fabb04313108ff3bfabebdee87bcee3289836ccdce47c37ea18a80","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.113Z","job_size_bytes":2592,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-d0417596062fc006193b4a78: done: 0.029938 sec","job_status":"done","queue_duration_s":0.227833,"scheduling_latency_s":0.22785,"redis_calls":7,"redis_duration_s":0.003504,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029938,"completed_at":"2026-09-08T04:27:04.370Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 250.649127] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.372Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"8f0f40eafae7101bdfff26c9","created_at":"2026-09-08T04:27:04.147Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-04761c72b8ab4b16","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.150Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-8f0f40eafae7101bdfff26c9: start","job_status":"start","queue_duration_s":0.221958,"scheduling_latency_s":0.221975}2155gitlab # [ 250.688815] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"8f0f40eafae7101bdfff26c9","created_at":"2026-09-08T04:27:04.147Z","trace_propagation_headers":{"sentry-trace":"f7e74bd55f25433e9a69c1c76de80407-04761c72b8ab4b16","baggage":"sentry-trace_id=f7e74bd55f25433e9a69c1c76de80407,sentry-release=Unknown"},"correlation_id":"01M1ZM8DTW2QGFCDXGHV5DEHV9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.150Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-8f0f40eafae7101bdfff26c9: done: 0.070703 sec","job_status":"done","queue_duration_s":0.221958,"scheduling_latency_s":0.221975,"redis_calls":11,"redis_duration_s":0.00172,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239975,"duration_s":0.070703,"completed_at":"2026-09-08T04:27:04.443Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 250.726754] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.444Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1cad341e7d12952df87809c8","created_at":"2026-09-08T04:27:04.267Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-676ed6d0c9f3473b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.270Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1cad341e7d12952df87809c8: start","job_status":"start","queue_duration_s":0.174276,"scheduling_latency_s":0.174292}2157gitlab # [ 250.727527] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.467Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1cad341e7d12952df87809c8","created_at":"2026-09-08T04:27:04.267Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-676ed6d0c9f3473b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.270Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1cad341e7d12952df87809c8: done: 0.02303 sec","job_status":"done","queue_duration_s":0.174276,"scheduling_latency_s":0.174292,"redis_calls":7,"redis_duration_s":0.000985,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02303,"completed_at":"2026-09-08T04:27:04.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158gitlab # [ 250.765331] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.468Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a5a202dbec1703e47d8459b4","created_at":"2026-09-08T04:27:04.332Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-676ed6d0c9f3473b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.334Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a5a202dbec1703e47d8459b4: start","job_status":"start","queue_duration_s":0.134096,"scheduling_latency_s":0.134111}2159gitlab # [ 250.766041] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.665Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a5a202dbec1703e47d8459b4","created_at":"2026-09-08T04:27:04.332Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-676ed6d0c9f3473b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.334Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2540,"job_status":"rescheduled","message":"PipelineProcessWorker JID-a5a202dbec1703e47d8459b4: rescheduled"}2160gitlab # [ 250.806135] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.671Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a5a202dbec1703e47d8459b4","created_at":"2026-09-08T04:27:04.332Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-676ed6d0c9f3473b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.334Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a5a202dbec1703e47d8459b4: done: 0.20273 sec","job_status":"done","queue_duration_s":0.134096,"scheduling_latency_s":0.134111,"gitaly_calls":3,"gitaly_duration_s":0.08235,"redis_calls":74,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00901,"redis_read_bytes":156,"redis_write_bytes":21173,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000658,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00065,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":25,"redis_queues_duration_s":0.002234,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14468,"redis_queues_metadata_calls":20,"redis_queues_metadata_duration_s":0.003076,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3301,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000195,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000366,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087568,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.178515146,"duration_s":0.20273,"completed_at":"2026-09-08T04:27:04.671Z","load_balancing_strategy":"primary","db_duration_s":0.02949,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2161gitlab # [ 250.845091] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.672Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4c1ea99f0d4b858a027e95ff","created_at":"2026-09-08T04:27:04.511Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.512Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4c1ea99f0d4b858a027e95ff: start","job_status":"start","queue_duration_s":0.159664,"scheduling_latency_s":0.159682}2162gitlab # [ 250.845700] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.698Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4c1ea99f0d4b858a027e95ff","created_at":"2026-09-08T04:27:04.511Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.512Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4c1ea99f0d4b858a027e95ff: done: 0.026672 sec","job_status":"done","queue_duration_s":0.159664,"scheduling_latency_s":0.159682,"redis_calls":7,"redis_duration_s":0.001062,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026672,"completed_at":"2026-09-08T04:27:04.698Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004421,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2163gitlab # [ 250.887125] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.699Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"373262efe241eb4576dda98c","created_at":"2026-09-08T04:27:04.596Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.599Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-373262efe241eb4576dda98c: start","job_status":"start","queue_duration_s":0.100268,"scheduling_latency_s":0.100284}2164gitlab # [ 250.887810] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.716Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"373262efe241eb4576dda98c","created_at":"2026-09-08T04:27:04.596Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.599Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-373262efe241eb4576dda98c: done: 0.01677 sec","job_status":"done","queue_duration_s":0.100268,"scheduling_latency_s":0.100284,"redis_calls":7,"redis_duration_s":0.000977,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01677,"completed_at":"2026-09-08T04:27:04.716Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2165gitlab # [ 250.933498] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"42ffcc1c532948d9803c5e18","created_at":"2026-09-08T04:27:04.605Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.606Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-42ffcc1c532948d9803c5e18: start","job_status":"start","queue_duration_s":0.110776,"scheduling_latency_s":0.110792}2166gitlab # [ 250.934076] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.735Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"42ffcc1c532948d9803c5e18","created_at":"2026-09-08T04:27:04.605Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.606Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-42ffcc1c532948d9803c5e18: done: 0.018389 sec","job_status":"done","queue_duration_s":0.110776,"scheduling_latency_s":0.110792,"redis_calls":7,"redis_duration_s":0.000986,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018389,"completed_at":"2026-09-08T04:27:04.735Z","load_balancing_strategy":"primary","db_duration_s":0.003425,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2167gitlab # [ 250.972642] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.736Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f096dce9c1e4f136082dc3e3","created_at":"2026-09-08T04:27:04.612Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.613Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f096dce9c1e4f136082dc3e3: start","job_status":"start","queue_duration_s":0.123485,"scheduling_latency_s":0.123499}2168gitlab # [ 250.978963] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.756Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f096dce9c1e4f136082dc3e3","created_at":"2026-09-08T04:27:04.612Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.613Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f096dce9c1e4f136082dc3e3: done: 0.019205 sec","job_status":"done","queue_duration_s":0.123485,"scheduling_latency_s":0.123499,"redis_calls":10,"redis_duration_s":0.001338,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00039,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019205,"completed_at":"2026-09-08T04:27:04.756Z","load_balancing_strategy":"primary","db_duration_s":0.002776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 251.025191] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.757Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"576f9eb41169f3235f2ffe48","created_at":"2026-09-08T04:27:04.613Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.614Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-576f9eb41169f3235f2ffe48: start","job_status":"start","queue_duration_s":0.142278,"scheduling_latency_s":0.142294}2170gitlab # [ 251.030664] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.773Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"576f9eb41169f3235f2ffe48","created_at":"2026-09-08T04:27:04.613Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.614Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-576f9eb41169f3235f2ffe48: done: 0.016083 sec","job_status":"done","queue_duration_s":0.142278,"scheduling_latency_s":0.142294,"redis_calls":7,"redis_duration_s":0.00099,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016083,"completed_at":"2026-09-08T04:27:04.773Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2171gitlab # [ 251.073676] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.774Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7cfaeb2cae3163ef48c82d28","created_at":"2026-09-08T04:27:04.615Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.616Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7cfaeb2cae3163ef48c82d28: start","job_status":"start","queue_duration_s":0.15751,"scheduling_latency_s":0.157524}2172gitlab # [ 251.114099] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.789Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7cfaeb2cae3163ef48c82d28","created_at":"2026-09-08T04:27:04.615Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.616Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7cfaeb2cae3163ef48c82d28: done: 0.015369 sec","job_status":"done","queue_duration_s":0.15751,"scheduling_latency_s":0.157524,"redis_calls":8,"redis_duration_s":0.001133,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015369,"completed_at":"2026-09-08T04:27:04.789Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2173gitlab # [ 251.117650] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.790Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4ac98a4b9e713508331d40ab","created_at":"2026-09-08T04:27:04.617Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.618Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4ac98a4b9e713508331d40ab: start","job_status":"start","queue_duration_s":0.172028,"scheduling_latency_s":0.172043}2174gitlab # [ 251.133950] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.804Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4ac98a4b9e713508331d40ab","created_at":"2026-09-08T04:27:04.617Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.618Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4ac98a4b9e713508331d40ab: done: 0.013981 sec","job_status":"done","queue_duration_s":0.172028,"scheduling_latency_s":0.172043,"redis_calls":7,"redis_duration_s":0.000943,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013981,"completed_at":"2026-09-08T04:27:04.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2175gitlab # [ 251.191962] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.805Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"6bcd0a8c96908d20fc1e4e14","created_at":"2026-09-08T04:27:04.619Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.620Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6bcd0a8c96908d20fc1e4e14: start","job_status":"start","queue_duration_s":0.185317,"scheduling_latency_s":0.185332}2176gitlab # [ 251.192661] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"6bcd0a8c96908d20fc1e4e14","created_at":"2026-09-08T04:27:04.619Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.620Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6bcd0a8c96908d20fc1e4e14: done: 0.021556 sec","job_status":"done","queue_duration_s":0.185317,"scheduling_latency_s":0.185332,"redis_calls":7,"redis_duration_s":0.001029,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021556,"completed_at":"2026-09-08T04:27:04.826Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177gitlab # [ 251.248803] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.827Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7aa4c2a4f323132d344c4372","created_at":"2026-09-08T04:27:04.622Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.623Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7aa4c2a4f323132d344c4372: start","job_status":"start","queue_duration_s":0.204376,"scheduling_latency_s":0.204392}2178gitlab # [ 251.249802] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.845Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7aa4c2a4f323132d344c4372","created_at":"2026-09-08T04:27:04.622Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.623Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7aa4c2a4f323132d344c4372: done: 0.017412 sec","job_status":"done","queue_duration_s":0.204376,"scheduling_latency_s":0.204392,"redis_calls":7,"redis_duration_s":0.0010609999999999999,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017412,"completed_at":"2026-09-08T04:27:04.845Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2179gitlab # [ 251.290640] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.846Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"df99c55fef7d70a78d7e5440","created_at":"2026-09-08T04:27:04.623Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.624Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-df99c55fef7d70a78d7e5440: start","job_status":"start","queue_duration_s":0.221522,"scheduling_latency_s":0.221538}2180gitlab # [ 251.291502] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.855Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"df99c55fef7d70a78d7e5440","created_at":"2026-09-08T04:27:04.623Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.624Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-df99c55fef7d70a78d7e5440: done: 0.009113 sec","job_status":"done","queue_duration_s":0.221522,"scheduling_latency_s":0.221538,"redis_calls":7,"redis_duration_s":0.000953,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009113,"completed_at":"2026-09-08T04:27:04.855Z","load_balancing_strategy":"primary","db_duration_s":0.000292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181gitlab # [ 251.334126] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6416f0792c2dd6010c302ff2","created_at":"2026-09-08T04:27:04.625Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.626Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6416f0792c2dd6010c302ff2: start","job_status":"start","queue_duration_s":0.230336,"scheduling_latency_s":0.230349}2182gitlab # [ 251.372871] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.870Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"6416f0792c2dd6010c302ff2","created_at":"2026-09-08T04:27:04.625Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.626Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6416f0792c2dd6010c302ff2: done: 0.014645 sec","job_status":"done","queue_duration_s":0.230336,"scheduling_latency_s":0.230349,"redis_calls":7,"redis_duration_s":0.001009,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014645,"completed_at":"2026-09-08T04:27:04.870Z","load_balancing_strategy":"primary","db_duration_s":0.002009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 251.373824] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.871Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"bac6bde6d79fc4a84f24a1ba","created_at":"2026-09-08T04:27:04.666Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.667Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bac6bde6d79fc4a84f24a1ba: start","job_status":"start","queue_duration_s":0.204487,"scheduling_latency_s":0.204501}2184gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'2185gitlab # [ 251.410949] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:04.887Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"bac6bde6d79fc4a84f24a1ba","created_at":"2026-09-08T04:27:04.666Z","trace_propagation_headers":{"sentry-trace":"e5773c51107d475ea0bf037e40cbd1ce-ce00a39af0014c6b","baggage":"sentry-trace_id=e5773c51107d475ea0bf037e40cbd1ce,sentry-release=Unknown"},"correlation_id":"01M1ZMA07W0DY8DE6857KH85G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:04.667Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bac6bde6d79fc4a84f24a1ba: done: 0.015925 sec","job_status":"done","queue_duration_s":0.204487,"scheduling_latency_s":0.204501,"redis_calls":8,"redis_duration_s":0.001145,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015925,"completed_at":"2026-09-08T04:27:04.887Z","load_balancing_strategy":"primary","db_duration_s":0.002011,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2186gitlab # [ 251.470114] nixbot[2663]: INFO: ::ffff:127.0.0.1:33772 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK2187gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.03 seconds)2188gitlab # [ 251.568074] gitaly[932]: time="2026-09-08T04:27:06.827Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA33R8A6W8HR4PK42SSY7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:06.817" grpc.time_ms=9.156 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2189gitlab # [ 251.569542] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 832190gitlab # [ 251.664039] gitaly[932]: time="2026-09-08T04:27:06.923Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA37BB9EFTM0RXFTWB7Y9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:06.899" pid=932 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2191gitlab # [ 251.664347] gitaly[932]: time="2026-09-08T04:27:06.924Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1167 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA37BB9EFTM0RXFTWB7Y9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:06.899" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2192gitlab # [ 251.665081] gitaly[932]: time="2026-09-08T04:27:06.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1167 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA37BB9EFTM0RXFTWB7Y9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-08T04:27:06.899" grpc.time_ms=24.583 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2193gitlab # [ 251.724402] gitlab-workhorse[833]: time="2026-09-08T04:27:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162194gitlab # [ 251.725037] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1242195gitlab # [ 251.814119] gitaly[932]: time="2026-09-08T04:27:07.074Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:37.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:07.072" grpc.time_ms=0.975 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2196gitlab # [ 251.841756] gitaly[932]: time="2026-09-08T04:27:07.098Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-08T04:27:37.086" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:07.086" grpc.time_ms=11.495 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2197gitlab # [ 251.879765] gitaly[932]: time="2026-09-08T04:27:07.139Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:17.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:07.130" grpc.time_ms=9.286 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2198gitlab # [ 251.924791] gitaly[932]: time="2026-09-08T04:27:07.184Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-08T04:27:17.063" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:27:07.184" grpc.time_ms=0.51 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2199gitlab # [ 251.963231] gitaly[932]: time="2026-09-08T04:27:07.223Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-08T04:27:17.090" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:07.210" grpc.time_ms=12.159 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 252.031468] gitaly[932]: time="2026-09-08T04:27:07.291Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:07.290" grpc.time_ms=1.216 pid=932 span.kind=server system=grpc2201gitlab # [ 252.076844] gitaly[932]: time="2026-09-08T04:27:07.336Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:07.335" grpc.time_ms=0.793 pid=932 span.kind=server system=grpc2202gitlab # [ 252.098578] gitaly[932]: time="2026-09-08T04:27:07.358Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ diskcache=5cb0238f-f507-4cef-b4b2-2319dc15eae9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T04:27:17.123" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:27:07.243" pid=932 ref=refs/keep-around/9b05cebdf8aa414871b7f2c72bf1e8b712287066 remote_ip="2001:db8:1::1" revision=9b05cebdf8aa414871b7f2c72bf1e8b712287066 span.kind=server system=grpc username=root2203gitlab # [ 252.099361] gitaly[932]: time="2026-09-08T04:27:07.358Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5216 command.oublock=8 command.real_time_ms=106 command.spawn_token_fork_ms=0 command.system_time_ms=59 command.user_time_ms=45 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-08T04:27:17.123" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-08T04:27:07.243" grpc.time_ms=114.731 pid=932 ref=refs/keep-around/9b05cebdf8aa414871b7f2c72bf1e8b712287066 remote_ip="2001:db8:1::1" revision=9b05cebdf8aa414871b7f2c72bf1e8b712287066 span.kind=server system=grpc username=root2204gitlab # [ 252.105404] gitaly[932]: time="2026-09-08T04:27:07.365Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:37.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:07.364" grpc.time_ms=0.452 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2205gitlab # [ 252.128466] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.387Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"78b960be054dc65f7375ecb9","created_at":"2026-09-08T04:27:07.376Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-0bc98d9a9ae4491b","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.384Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-78b960be054dc65f7375ecb9: start","job_status":"start","queue_duration_s":0.002965,"scheduling_latency_s":0.003035}2206gitlab # [ 252.167301] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.426Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"78b960be054dc65f7375ecb9","created_at":"2026-09-08T04:27:07.376Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-0bc98d9a9ae4491b","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.384Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-78b960be054dc65f7375ecb9: done: 0.039802 sec","job_status":"done","queue_duration_s":0.002965,"scheduling_latency_s":0.003035,"redis_calls":7,"redis_duration_s":0.002141,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001619,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039802,"completed_at":"2026-09-08T04:27:07.426Z","load_balancing_strategy":"primary","db_duration_s":0.005661,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2207gitlab # [ 252.206422] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.465Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2bf6712b620878c79b0f0335","created_at":"2026-09-08T04:27:07.386Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-0bc98d9a9ae4491b","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.390Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2bf6712b620878c79b0f0335: start","job_status":"start","queue_duration_s":0.075119,"scheduling_latency_s":0.075168}2208gitlab # [ 252.249398] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4772209gitlab # [ 252.252441] nixbot[2663]: {"ts": "2026-09-08T04:27:07.510Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 \"HTTP/1.1 201 Created\""}2210gitlab # [ 252.263622] nixbot[2663]: {"ts": "2026-09-08T04:27:07.523Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-9b05cebdf8aa-5"}2211gitlab # [ 252.296323] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.555Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2bf6712b620878c79b0f0335","created_at":"2026-09-08T04:27:07.386Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-0bc98d9a9ae4491b","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.390Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2bf6712b620878c79b0f0335: done: 0.089936 sec","job_status":"done","queue_duration_s":0.075119,"scheduling_latency_s":0.075168,"redis_calls":10,"redis_duration_s":0.0027670000000000004,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089936,"completed_at":"2026-09-08T04:27:07.555Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019979,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 252.299392] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.559Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"18223b38f9539b16c59d92a0","created_at":"2026-09-08T04:27:07.481Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-0bc98d9a9ae4491b","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.488Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-18223b38f9539b16c59d92a0: start","job_status":"start","queue_duration_s":0.071103,"scheduling_latency_s":0.071139}2213gitlab # [ 252.376418] gitaly[932]: time="2026-09-08T04:27:07.636Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:37.634" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:07.634" grpc.time_ms=1.238 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2214gitlab # [ 252.419968] gitaly[932]: time="2026-09-08T04:27:07.680Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA3CQGPKZ4T2TYM2TKPVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:37.678" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:07.678" grpc.time_ms=0.89 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2215gitlab # [ 252.431513] nixbot[2663]: {"ts": "2026-09-08T04:27:07.680Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-9b05cebdf8aa-5", "argv0": "bwrap"}2216gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066'2217gitlab # [ 252.443123] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.702Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"18223b38f9539b16c59d92a0","created_at":"2026-09-08T04:27:07.481Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-0bc98d9a9ae4491b","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.488Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-18223b38f9539b16c59d92a0: done: 0.143179 sec","job_status":"done","queue_duration_s":0.071103,"scheduling_latency_s":0.071139,"gitaly_calls":2,"gitaly_duration_s":0.012203,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.016495,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001099,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001178,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000275,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003421,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000264,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.007932,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001227,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091246,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10729995,"duration_s":0.143179,"completed_at":"2026-09-08T04:27:07.702Z","load_balancing_strategy":"primary","db_duration_s":0.024661,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2218gitlab # [ 252.449760] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.706Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"be2320af7ed28ef63ca74851","created_at":"2026-09-08T04:27:07.621Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-d90140595403406a","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.623Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-be2320af7ed28ef63ca74851: start","job_status":"start","queue_duration_s":0.082933,"scheduling_latency_s":0.08295}2219gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066', in 0.03 seconds)2220(finished: subtest: push delivers the webhook, in 50.97 seconds)2221subtest: eval and build post commit statuses2222gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2223gitlab # [ 252.470789] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:07.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"be2320af7ed28ef63ca74851","created_at":"2026-09-08T04:27:07.621Z","trace_propagation_headers":{"sentry-trace":"c05fdcd8f04f41b3b038d8c9f6d5f399-d90140595403406a","baggage":"sentry-trace_id=c05fdcd8f04f41b3b038d8c9f6d5f399,sentry-release=Unknown"},"correlation_id":"01M1ZMA3CQGPKZ4T2TYM2TKPVQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:07.623Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-be2320af7ed28ef63ca74851: done: 0.024301 sec","job_status":"done","queue_duration_s":0.082933,"scheduling_latency_s":0.08295,"redis_calls":7,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024301,"completed_at":"2026-09-08T04:27:07.730Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 252.530106] nixbot[2663]: INFO: ::ffff:127.0.0.1:33788 - "GET /api/repos/gitlab/root/test-flake/builds?commit=9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 200 OK2225gitlab # [ 252.572370] gitaly[932]: time="2026-09-08T04:27:07.831Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA44FXSNWTSZWPB5JZGVD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:37.329" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:07.830" grpc.time_ms=0.891 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226gitlab # [ 252.586496] nix-daemon[2829]: accepted connection from pid 3653, user nixbot2227gitlab # [ 252.597734] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 652228gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.09 seconds)2229gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2230gitlab # [ 253.697316] gitaly[932]: time="2026-09-08T04:27:08.956Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA570H9TY6641P4JNBH9X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:38.454" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:08.955" grpc.time_ms=1.095 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2231gitlab # [ 253.723001] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 852232gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.13 seconds)2233gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2234gitlab # [ 254.856706] gitaly[932]: time="2026-09-08T04:27:10.115Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA6AA2Q35KV5BQCV3WVDQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:40.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:10.114" grpc.time_ms=0.979 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2235gitlab # [ 254.935735] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1682236gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.21 seconds)2237gitlab # [ 255.315933] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"021bad47c8d994c4328e7bc2","created_at":"2026-09-08T04:27:10.555Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86be88ca9fa51008f2e655a175841770","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.571Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-021bad47c8d994c4328e7bc2: start","job_status":"start","queue_duration_s":0.003154,"scheduling_latency_s":0.003208}2238gitlab # [ 255.502448] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"021bad47c8d994c4328e7bc2","created_at":"2026-09-08T04:27:10.555Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86be88ca9fa51008f2e655a175841770","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.571Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-021bad47c8d994c4328e7bc2: done: 0.187535 sec","job_status":"done","queue_duration_s":0.003154,"scheduling_latency_s":0.003208,"redis_calls":11,"redis_duration_s":0.010535,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002887,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001118857,"exclusive_lock_hold_duration_s":0.005829791,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.187535,"completed_at":"2026-09-08T04:27:10.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2239gitlab # [ 255.578692] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"1640d168f9763b087ae67aec","created_at":"2026-09-08T04:27:10.612Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02d85253e31ea1073020570b50a04959","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T04:27:10.627Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-1640d168f9763b087ae67aec: start","job_status":"start","queue_duration_s":0.137446,"scheduling_latency_s":0.137469}2240gitlab # [ 255.590270] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.822Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"1640d168f9763b087ae67aec","created_at":"2026-09-08T04:27:10.612Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02d85253e31ea1073020570b50a04959","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-08T04:27:10.627Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-1640d168f9763b087ae67aec: done: 0.057843 sec","job_status":"done","queue_duration_s":0.137446,"scheduling_latency_s":0.137469,"redis_calls":9,"redis_duration_s":0.007043000000000001,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001204,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001386,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057843,"completed_at":"2026-09-08T04:27:10.822Z","load_balancing_strategy":"primary","db_duration_s":0.018917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 255.627987] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4a900be04ce7bd9dbf0b8402","created_at":"2026-09-08T04:27:10.662Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06a82272a87f2d9bbb59ece433f737c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.781Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4a900be04ce7bd9dbf0b8402: start","job_status":"start","queue_duration_s":0.046614,"scheduling_latency_s":0.046638}2242gitlab # [ 255.641591] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.879Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2243gitlab # [ 255.642276] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.895Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4a900be04ce7bd9dbf0b8402","created_at":"2026-09-08T04:27:10.662Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06a82272a87f2d9bbb59ece433f737c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.781Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4a900be04ce7bd9dbf0b8402: done: 0.066733 sec","job_status":"done","queue_duration_s":0.046614,"scheduling_latency_s":0.046638,"redis_calls":9,"redis_duration_s":0.012668,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004211,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002219,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066733,"completed_at":"2026-09-08T04:27:10.895Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 255.683408] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"bc14aef9578df4d8bca90b4c","created_at":"2026-09-08T04:27:10.809Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f352043d9c2b022af26058956a31eba9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.827Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-bc14aef9578df4d8bca90b4c: start","job_status":"start","queue_duration_s":0.08332,"scheduling_latency_s":0.083357}2245gitlab # [ 255.739112] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:10.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"bc14aef9578df4d8bca90b4c","created_at":"2026-09-08T04:27:10.809Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f352043d9c2b022af26058956a31eba9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.827Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-bc14aef9578df4d8bca90b4c: done: 0.087593 sec","job_status":"done","queue_duration_s":0.08332,"scheduling_latency_s":0.083357,"redis_calls":12,"redis_duration_s":0.01168,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005264,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004023,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.019527621,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.087593,"completed_at":"2026-09-08T04:27:10.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020948,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2246gitlab # [ 255.744073] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5a5f4555229e5d6c64895ede","created_at":"2026-09-08T04:27:10.854Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f70a2959bce5ec21bd706a37a30eeae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.867Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5a5f4555229e5d6c64895ede: start","job_status":"start","queue_duration_s":0.136248,"scheduling_latency_s":0.136293}2247gitlab # [ 255.829433] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.089Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2248gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2249gitlab # [ 255.882610] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5a5f4555229e5d6c64895ede","created_at":"2026-09-08T04:27:10.854Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f70a2959bce5ec21bd706a37a30eeae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.867Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5a5f4555229e5d6c64895ede: done: 0.138201 sec","job_status":"done","queue_duration_s":0.136248,"scheduling_latency_s":0.136293,"redis_calls":9,"redis_duration_s":0.036691,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002083,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.027063,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.138201,"completed_at":"2026-09-08T04:27:11.141Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250gitlab # [ 255.960774] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"878c81bc16baa6a18fa94fba","created_at":"2026-09-08T04:27:10.879Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.892Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-878c81bc16baa6a18fa94fba: start","job_status":"start","queue_duration_s":0.326887,"scheduling_latency_s":0.326951}2251gitlab # [ 256.060630] gitaly[932]: time="2026-09-08T04:27:11.319Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA7GH0682M8P6VQZXK89V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:41.217" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:11.318" grpc.time_ms=1.035 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2252gitlab # [ 256.086369] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 952253gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.15 seconds)2254gitlab # [ 256.091255] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"878c81bc16baa6a18fa94fba","created_at":"2026-09-08T04:27:10.879Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.892Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-878c81bc16baa6a18fa94fba: done: 0.128013 sec","job_status":"done","queue_duration_s":0.326887,"scheduling_latency_s":0.326951,"redis_calls":16,"redis_duration_s":0.048166,"redis_read_bytes":18,"redis_write_bytes":4343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_calls":2,"redis_queues_duration_s":0.000703,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.036842,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.008345,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.128013,"completed_at":"2026-09-08T04:27:11.347Z","load_balancing_strategy":"primary","db_duration_s":0.01024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 256.113469] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bb9feb25447b52779a35353d","created_at":"2026-09-08T04:27:10.909Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b71dfc1184b80ef27c6b025eab6fbe5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.922Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bb9feb25447b52779a35353d: start","job_status":"start","queue_duration_s":0.426797,"scheduling_latency_s":0.426834}2256gitlab # [ 256.144032] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bb9feb25447b52779a35353d","created_at":"2026-09-08T04:27:10.909Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b71dfc1184b80ef27c6b025eab6fbe5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.922Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bb9feb25447b52779a35353d: done: 0.05433 sec","job_status":"done","queue_duration_s":0.426797,"scheduling_latency_s":0.426834,"redis_calls":13,"redis_duration_s":0.004097,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000872,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000584432,"exclusive_lock_hold_duration_s":0.016827279,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.05433,"completed_at":"2026-09-08T04:27:11.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 256.145092] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2cefdf0a62f0beecc15f2567","created_at":"2026-09-08T04:27:10.932Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cac9c8b66c7cd8aafdf3c5d5b3afc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.938Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2cefdf0a62f0beecc15f2567: start","job_status":"start","queue_duration_s":0.466338,"scheduling_latency_s":0.466436}2258gitlab # [ 256.205921] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2cefdf0a62f0beecc15f2567","created_at":"2026-09-08T04:27:10.932Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cac9c8b66c7cd8aafdf3c5d5b3afc4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.938Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2cefdf0a62f0beecc15f2567: done: 0.060845 sec","job_status":"done","queue_duration_s":0.466338,"scheduling_latency_s":0.466436,"redis_calls":7,"redis_duration_s":0.001044,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060845,"completed_at":"2026-09-08T04:27:11.465Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259gitlab # [ 256.243066] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e6f13e6eb4918673357b57e2","created_at":"2026-09-08T04:27:10.949Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91a4e80f2372d0d8c04056f1d4afaa3d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T04:27:10.960Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e6f13e6eb4918673357b57e2: start","job_status":"start","queue_duration_s":0.542142,"scheduling_latency_s":0.542165}2260gitlab # [ 256.263305] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.523Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2261gitlab # [ 256.273395] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e6f13e6eb4918673357b57e2","created_at":"2026-09-08T04:27:10.949Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91a4e80f2372d0d8c04056f1d4afaa3d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-08T04:27:10.960Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e6f13e6eb4918673357b57e2: done: 0.029919 sec","job_status":"done","queue_duration_s":0.542142,"scheduling_latency_s":0.542165,"redis_calls":7,"redis_duration_s":0.003664,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003212,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029919,"completed_at":"2026-09-08T04:27:11.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab # [ 256.310083] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d8694fef79705f10ac0baaec","created_at":"2026-09-08T04:27:10.970Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0227b75d7bb9cab382bced1bcc9c77bf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.976Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d8694fef79705f10ac0baaec: start","job_status":"start","queue_duration_s":0.593441,"scheduling_latency_s":0.593481}2263gitlab # [ 256.686110] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.944Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d8694fef79705f10ac0baaec","created_at":"2026-09-08T04:27:10.970Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0227b75d7bb9cab382bced1bcc9c77bf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:10.976Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d8694fef79705f10ac0baaec: done: 0.375476 sec","job_status":"done","queue_duration_s":0.593441,"scheduling_latency_s":0.593481,"redis_calls":562,"redis_duration_s":0.113823,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.1127,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.282815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.375476,"completed_at":"2026-09-08T04:27:11.944Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264gitlab # [ 256.687479] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:11.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2c33202abab68b0438e94d52","created_at":"2026-09-08T04:27:10.993Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29ec410b3213423109b385756d123703","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.000Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2c33202abab68b0438e94d52: start","job_status":"start","queue_duration_s":0.945452,"scheduling_latency_s":0.945501}2265gitlab # [ 256.912822] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2c33202abab68b0438e94d52","created_at":"2026-09-08T04:27:10.993Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29ec410b3213423109b385756d123703","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.000Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2c33202abab68b0438e94d52: done: 0.226339 sec","job_status":"done","queue_duration_s":0.945452,"scheduling_latency_s":0.945501,"redis_calls":9,"redis_duration_s":0.0024070000000000003,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000502019,"exclusive_lock_hold_duration_s":0.165895386,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.226339,"completed_at":"2026-09-08T04:27:12.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2266gitlab # [ 256.914293] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a8898d86f53549ac7a8f42e7","created_at":"2026-09-08T04:27:11.057Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d54985732b8218d91ee3f2cc3673273d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.063Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a8898d86f53549ac7a8f42e7: start","job_status":"start","queue_duration_s":1.110807,"scheduling_latency_s":1.110823}2267gitlab # [ 256.990355] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a8898d86f53549ac7a8f42e7","created_at":"2026-09-08T04:27:11.057Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d54985732b8218d91ee3f2cc3673273d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.063Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a8898d86f53549ac7a8f42e7: done: 0.009175 sec","job_status":"done","queue_duration_s":1.110807,"scheduling_latency_s":1.110823,"redis_calls":7,"redis_duration_s":0.001578,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009175,"completed_at":"2026-09-08T04:27:12.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268gitlab # [ 257.033729] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c5eabb6f600cef1861a15bb7","created_at":"2026-09-08T04:27:11.077Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.084Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c5eabb6f600cef1861a15bb7: start","job_status":"start","queue_duration_s":1.099813,"scheduling_latency_s":1.099828}2269gitlab # [ 257.034860] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c5eabb6f600cef1861a15bb7","created_at":"2026-09-08T04:27:11.077Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.084Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c5eabb6f600cef1861a15bb7: done: 0.057307 sec","job_status":"done","queue_duration_s":1.099813,"scheduling_latency_s":1.099828,"redis_calls":60,"redis_duration_s":0.013571000000000001,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000718,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.012226,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057307,"completed_at":"2026-09-08T04:27:12.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2270gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2271gitlab # [ 257.084667] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"05447a77af82a116d87e439e","created_at":"2026-09-08T04:27:11.125Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ab23f2045c87963cf37614cf60c6f5a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T04:27:11.138Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-05447a77af82a116d87e439e: start","job_status":"start","queue_duration_s":1.104355,"scheduling_latency_s":1.104372}2272gitlab # [ 257.085608] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.248Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2273gitlab # [ 257.086375] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.251Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"05447a77af82a116d87e439e","created_at":"2026-09-08T04:27:11.125Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ab23f2045c87963cf37614cf60c6f5a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-08T04:27:11.138Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-05447a77af82a116d87e439e: done: 0.009101 sec","job_status":"done","queue_duration_s":1.104355,"scheduling_latency_s":1.104372,"redis_calls":7,"redis_duration_s":0.00207,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009101,"completed_at":"2026-09-08T04:27:12.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274gitlab # [ 257.133447] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"edcad5587b063c12fbf7aefb","created_at":"2026-09-08T04:27:11.221Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a099f7aa92485e574549e6bf38bd4045","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.229Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-edcad5587b063c12fbf7aefb: start","job_status":"start","queue_duration_s":1.023905,"scheduling_latency_s":1.023918}2275gitlab # [ 257.134603] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"edcad5587b063c12fbf7aefb","created_at":"2026-09-08T04:27:11.221Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a099f7aa92485e574549e6bf38bd4045","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.229Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-edcad5587b063c12fbf7aefb: done: 0.036437 sec","job_status":"done","queue_duration_s":1.023905,"scheduling_latency_s":1.023918,"redis_calls":9,"redis_duration_s":0.002005,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000500063,"exclusive_lock_hold_duration_s":0.01427891,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.036437,"completed_at":"2026-09-08T04:27:12.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 257.185970] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7d7dae85dda8dc5fc150070d","created_at":"2026-09-08T04:27:11.268Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18ada695802a7650d5ca0080c9abca14","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.275Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7d7dae85dda8dc5fc150070d: start","job_status":"start","queue_duration_s":1.014604,"scheduling_latency_s":1.014618}2277gitlab # [ 257.186618] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7d7dae85dda8dc5fc150070d","created_at":"2026-09-08T04:27:11.268Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18ada695802a7650d5ca0080c9abca14","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.275Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7d7dae85dda8dc5fc150070d: done: 0.017477 sec","job_status":"done","queue_duration_s":1.014604,"scheduling_latency_s":1.014618,"redis_calls":7,"redis_duration_s":0.001794,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.017477,"completed_at":"2026-09-08T04:27:12.308Z","load_balancing_strategy":"primary","db_duration_s":0.007695,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2278gitlab # [ 257.232917] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a862cc228a0125c1202e458d","created_at":"2026-09-08T04:27:11.288Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2715fabb09c143ddddec44b8c4c022be","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.292Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a862cc228a0125c1202e458d: start","job_status":"start","queue_duration_s":1.016598,"scheduling_latency_s":1.016612}2279gitlab # [ 257.233649] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a862cc228a0125c1202e458d","created_at":"2026-09-08T04:27:11.288Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2715fabb09c143ddddec44b8c4c022be","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.292Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a862cc228a0125c1202e458d: done: 0.009746 sec","job_status":"done","queue_duration_s":1.016598,"scheduling_latency_s":1.016612,"redis_calls":7,"redis_duration_s":0.001418,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009746,"completed_at":"2026-09-08T04:27:12.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2280gitlab # [ 257.246582] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"70df3351549fd7d061624a9f","created_at":"2026-09-08T04:27:11.311Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dfdcfe58154a4f29221ec6856bdac527","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.316Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-70df3351549fd7d061624a9f: start","job_status":"start","queue_duration_s":1.003605,"scheduling_latency_s":1.003618}2281gitlab # [ 257.294181] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.324Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2282gitlab # [ 257.294841] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"70df3351549fd7d061624a9f","created_at":"2026-09-08T04:27:11.311Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dfdcfe58154a4f29221ec6856bdac527","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.316Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-70df3351549fd7d061624a9f: done: 0.007906 sec","job_status":"done","queue_duration_s":1.003605,"scheduling_latency_s":1.003618,"redis_calls":7,"redis_duration_s":0.001278,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007906,"completed_at":"2026-09-08T04:27:12.327Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 257.298076] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.328Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"63c375ae00ed8f2c46fc55c8","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-63c375ae00ed8f2c46fc55c8: start","job_status":"start","queue_duration_s":1.011927,"scheduling_latency_s":1.011941}2284gitlab # [ 257.308876] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.336Z","message":"No existing merge request to be cleaned up."}2285gitlab # [ 257.347843] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.341Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"63c375ae00ed8f2c46fc55c8","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-63c375ae00ed8f2c46fc55c8: done: 0.012864 sec","job_status":"done","queue_duration_s":1.011927,"scheduling_latency_s":1.011941,"redis_calls":10,"redis_duration_s":0.0020870000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012864,"completed_at":"2026-09-08T04:27:12.341Z","load_balancing_strategy":"primary","db_duration_s":0.001806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2286gitlab # [ 257.348876] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.342Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c17bb79498bd1d81bdf7b0b1","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c17bb79498bd1d81bdf7b0b1: start","job_status":"start","queue_duration_s":1.025868,"scheduling_latency_s":1.025882}2287gitlab # [ 257.354801] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.349Z","message":"No existing merge request to be cleaned up."}2288gitlab # [ 257.356581] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.354Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c17bb79498bd1d81bdf7b0b1","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c17bb79498bd1d81bdf7b0b1: done: 0.011806 sec","job_status":"done","queue_duration_s":1.025868,"scheduling_latency_s":1.025882,"redis_calls":10,"redis_duration_s":0.00207,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011806,"completed_at":"2026-09-08T04:27:12.354Z","load_balancing_strategy":"primary","db_duration_s":0.00144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2289gitlab # [ 257.401675] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.355Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a709cd0e04154f4c58d07439","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a709cd0e04154f4c58d07439: start","job_status":"start","queue_duration_s":1.038741,"scheduling_latency_s":1.038755}2290gitlab # [ 257.402458] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.362Z","message":"No existing merge request to be cleaned up."}2291gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.38 seconds)2292gitlab # [ 257.403075] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.371Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a709cd0e04154f4c58d07439","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a709cd0e04154f4c58d07439: done: 0.015822 sec","job_status":"done","queue_duration_s":1.038741,"scheduling_latency_s":1.038755,"redis_calls":10,"redis_duration_s":0.005489,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00359,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015822,"completed_at":"2026-09-08T04:27:12.371Z","load_balancing_strategy":"primary","db_duration_s":0.001621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293gitlab # [ 257.460125] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.372Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac8db558c82a2bb165653eed","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac8db558c82a2bb165653eed: start","job_status":"start","queue_duration_s":1.055758,"scheduling_latency_s":1.055772}2294gitlab # [ 257.460766] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.379Z","message":"No existing merge request to be cleaned up."}2295gitlab # [ 257.461511] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.385Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac8db558c82a2bb165653eed","created_at":"2026-09-08T04:27:11.293Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-73f3fc9647aa48a1","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0b50d5a3d57d34c1fcfbe9d096b268ec","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.317Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac8db558c82a2bb165653eed: done: 0.012379 sec","job_status":"done","queue_duration_s":1.055758,"scheduling_latency_s":1.055772,"redis_calls":10,"redis_duration_s":0.0021609999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012379,"completed_at":"2026-09-08T04:27:12.385Z","load_balancing_strategy":"primary","db_duration_s":0.001641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 257.485580] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2ec090c83b4546829bc0e6dd","created_at":"2026-09-08T04:27:11.330Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9caf7c5d101a98a4563f96f6ed1e5a71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.340Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2ec090c83b4546829bc0e6dd: start","job_status":"start","queue_duration_s":1.046119,"scheduling_latency_s":1.046143}2297gitlab # [ 257.530063] gitaly[932]: time="2026-09-08T04:27:12.708Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA8WZ22PXJP8C0NBHD30Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:42.606" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:12.707" grpc.time_ms=0.473 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2298gitlab # [ 257.531023] nix-daemon[2829]: accepted connection from pid 3690, user nixbot2299gitlab # [ 257.531964] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 522300gitlab # [ 257.534788] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.397Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2ec090c83b4546829bc0e6dd","created_at":"2026-09-08T04:27:11.330Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9caf7c5d101a98a4563f96f6ed1e5a71","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.340Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2ec090c83b4546829bc0e6dd: done: 0.011389 sec","job_status":"done","queue_duration_s":1.046119,"scheduling_latency_s":1.046143,"redis_calls":9,"redis_duration_s":0.001983,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000428266,"exclusive_lock_hold_duration_s":0.002498362,"duration_s":0.011389,"completed_at":"2026-09-08T04:27:12.397Z","load_balancing_strategy":"primary","db_duration_s":0.001074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2301gitlab # [ 257.582926] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f73c22c69d87d992d1cd4556","created_at":"2026-09-08T04:27:11.351Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa4d804206f80b7e701b1375a7d5c8ab","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.354Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f73c22c69d87d992d1cd4556: start","job_status":"start","queue_duration_s":1.044204,"scheduling_latency_s":1.044216}2302gitlab # [ 257.583684] nix-daemon[2829]: accepted connection from pid 3653, user nixbot2303gitlab # [ 257.584418] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.411Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f73c22c69d87d992d1cd4556","created_at":"2026-09-08T04:27:11.351Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa4d804206f80b7e701b1375a7d5c8ab","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.354Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f73c22c69d87d992d1cd4556: done: 0.013254 sec","job_status":"done","queue_duration_s":1.044204,"scheduling_latency_s":1.044216,"redis_calls":7,"redis_duration_s":0.001452,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013254,"completed_at":"2026-09-08T04:27:12.411Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003353,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2304gitlab # [ 257.596641] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"23e0c75f703525a8ac78794d","created_at":"2026-09-08T04:27:11.361Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa3f909cb4ddd8beefab37346bb46c37","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.367Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-23e0c75f703525a8ac78794d: start","job_status":"start","queue_duration_s":1.045291,"scheduling_latency_s":1.045305}2305gitlab # [ 257.635882] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.417Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2306gitlab # [ 257.636617] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"23e0c75f703525a8ac78794d","created_at":"2026-09-08T04:27:11.361Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-238c52c165f64671","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa3f909cb4ddd8beefab37346bb46c37","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:11.367Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-23e0c75f703525a8ac78794d: done: 0.007999 sec","job_status":"done","queue_duration_s":1.045291,"scheduling_latency_s":1.045305,"redis_calls":7,"redis_duration_s":0.001372,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007999,"completed_at":"2026-09-08T04:27:12.421Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2307gitlab # [ 257.647755] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e90d174864922e4e8dd9d337","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e90d174864922e4e8dd9d337: start","job_status":"start","queue_duration_s":0.185979,"scheduling_latency_s":0.185992}2308gitlab # [ 257.691107] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e90d174864922e4e8dd9d337","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e90d174864922e4e8dd9d337: done: 0.010298 sec","job_status":"done","queue_duration_s":0.185979,"scheduling_latency_s":0.185992,"redis_calls":13,"redis_duration_s":0.002464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001714,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010298,"completed_at":"2026-09-08T04:27:12.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2309gitlab # [ 257.730092] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3b0f17ff707ad0176347035","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3b0f17ff707ad0176347035: start","job_status":"start","queue_duration_s":0.197323,"scheduling_latency_s":0.197337}2310gitlab # [ 257.730824] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3b0f17ff707ad0176347035","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3b0f17ff707ad0176347035: done: 0.01005 sec","job_status":"done","queue_duration_s":0.197323,"scheduling_latency_s":0.197337,"redis_calls":13,"redis_duration_s":0.00244,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001687,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01005,"completed_at":"2026-09-08T04:27:12.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab # [ 257.738864] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b63dd6ef8ff0d4e272f8210d","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b63dd6ef8ff0d4e272f8210d: start","job_status":"start","queue_duration_s":0.2084,"scheduling_latency_s":0.208413}2312gitlab # [ 257.750534] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b63dd6ef8ff0d4e272f8210d","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b63dd6ef8ff0d4e272f8210d: done: 0.022615 sec","job_status":"done","queue_duration_s":0.2084,"scheduling_latency_s":0.208413,"redis_calls":13,"redis_duration_s":0.003575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001205,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022615,"completed_at":"2026-09-08T04:27:12.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 257.798239] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83420608b4e640f307b33b7c","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83420608b4e640f307b33b7c: start","job_status":"start","queue_duration_s":0.233243,"scheduling_latency_s":0.233273}2314gitlab # [ 257.798878] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83420608b4e640f307b33b7c","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83420608b4e640f307b33b7c: done: 0.019259 sec","job_status":"done","queue_duration_s":0.233243,"scheduling_latency_s":0.233273,"redis_calls":13,"redis_duration_s":0.003259,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019259,"completed_at":"2026-09-08T04:27:12.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2315gitlab # [ 257.843463] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0448f155f165217b8aaa2a09","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0448f155f165217b8aaa2a09: start","job_status":"start","queue_duration_s":0.253992,"scheduling_latency_s":0.254016}2316gitlab # [ 257.844204] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0448f155f165217b8aaa2a09","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0448f155f165217b8aaa2a09: done: 0.010752 sec","job_status":"done","queue_duration_s":0.253992,"scheduling_latency_s":0.254016,"redis_calls":13,"redis_duration_s":0.0023769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010752,"completed_at":"2026-09-08T04:27:12.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2317gitlab # [ 257.902956] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f00bda4b1ba40ff4976278f","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f00bda4b1ba40ff4976278f: start","job_status":"start","queue_duration_s":0.265862,"scheduling_latency_s":0.265875}2318gitlab # [ 257.903701] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f00bda4b1ba40ff4976278f","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f00bda4b1ba40ff4976278f: done: 0.050126 sec","job_status":"done","queue_duration_s":0.265862,"scheduling_latency_s":0.265875,"redis_calls":13,"redis_duration_s":0.0026490000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.050126,"completed_at":"2026-09-08T04:27:12.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2319gitlab # [ 257.955860] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f7be2ed53f6501304483eed","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f7be2ed53f6501304483eed: start","job_status":"start","queue_duration_s":0.317449,"scheduling_latency_s":0.317467}2320gitlab # [ 257.956644] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f7be2ed53f6501304483eed","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f7be2ed53f6501304483eed: done: 0.011394 sec","job_status":"done","queue_duration_s":0.317449,"scheduling_latency_s":0.317467,"redis_calls":13,"redis_duration_s":0.0024159999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001602,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011394,"completed_at":"2026-09-08T04:27:12.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321gitlab # [ 257.965398] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d10e0de9d458046999b327cd","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d10e0de9d458046999b327cd: start","job_status":"start","queue_duration_s":0.330198,"scheduling_latency_s":0.330211}2322gitlab # [ 257.976691] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d10e0de9d458046999b327cd","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d10e0de9d458046999b327cd: done: 0.016676 sec","job_status":"done","queue_duration_s":0.330198,"scheduling_latency_s":0.330211,"redis_calls":13,"redis_duration_s":0.005711,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016676,"completed_at":"2026-09-08T04:27:12.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2323gitlab # [ 258.035979] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f03cba03fd39a41e6e5ce814","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f03cba03fd39a41e6e5ce814: start","job_status":"start","queue_duration_s":0.348243,"scheduling_latency_s":0.348264}2324gitlab # [ 258.036730] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f03cba03fd39a41e6e5ce814","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f03cba03fd39a41e6e5ce814: done: 0.022628 sec","job_status":"done","queue_duration_s":0.348243,"scheduling_latency_s":0.348264,"redis_calls":13,"redis_duration_s":0.003283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022628,"completed_at":"2026-09-08T04:27:12.606Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2325gitlab # [ 258.055824] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a387cdace0983b17456e0c7","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a387cdace0983b17456e0c7: start","job_status":"start","queue_duration_s":0.373592,"scheduling_latency_s":0.373628}2326gitlab # [ 258.094411] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a387cdace0983b17456e0c7","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a387cdace0983b17456e0c7: done: 0.019208 sec","job_status":"done","queue_duration_s":0.373592,"scheduling_latency_s":0.373628,"redis_calls":13,"redis_duration_s":0.0028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019208,"completed_at":"2026-09-08T04:27:12.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2327gitlab # [ 258.101573] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df3e86e22be6751bfd8bb61e","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df3e86e22be6751bfd8bb61e: start","job_status":"start","queue_duration_s":0.394816,"scheduling_latency_s":0.39484}2328gitlab # [ 258.144270] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df3e86e22be6751bfd8bb61e","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df3e86e22be6751bfd8bb61e: done: 0.013953 sec","job_status":"done","queue_duration_s":0.394816,"scheduling_latency_s":0.39484,"redis_calls":13,"redis_duration_s":0.0019399999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013953,"completed_at":"2026-09-08T04:27:12.644Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2329gitlab # [ 258.147189] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4edfe066958bcdeb3018d65","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4edfe066958bcdeb3018d65: start","job_status":"start","queue_duration_s":0.409592,"scheduling_latency_s":0.409606}2330gitlab # [ 258.187359] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4edfe066958bcdeb3018d65","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4edfe066958bcdeb3018d65: done: 0.009289 sec","job_status":"done","queue_duration_s":0.409592,"scheduling_latency_s":0.409606,"redis_calls":13,"redis_duration_s":0.001501,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009289,"completed_at":"2026-09-08T04:27:12.654Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2331gitlab # [ 258.191616] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"905b45c76a42ae6875d161de","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-905b45c76a42ae6875d161de: start","job_status":"start","queue_duration_s":0.420115,"scheduling_latency_s":0.420154}2332gitlab # [ 258.233583] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"905b45c76a42ae6875d161de","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-905b45c76a42ae6875d161de: done: 0.012887 sec","job_status":"done","queue_duration_s":0.420115,"scheduling_latency_s":0.420154,"redis_calls":13,"redis_duration_s":0.001887,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012887,"completed_at":"2026-09-08T04:27:12.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2333gitlab # [ 258.236432] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b97c1e6f952c4c355fb3f310","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b97c1e6f952c4c355fb3f310: start","job_status":"start","queue_duration_s":0.43412,"scheduling_latency_s":0.434165}2334gitlab # [ 258.276524] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b97c1e6f952c4c355fb3f310","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b97c1e6f952c4c355fb3f310: done: 0.010437 sec","job_status":"done","queue_duration_s":0.43412,"scheduling_latency_s":0.434165,"redis_calls":13,"redis_duration_s":0.001662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010437,"completed_at":"2026-09-08T04:27:12.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2335gitlab # [ 258.280996] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ace4dbd94c78cea4b68781e5","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ace4dbd94c78cea4b68781e5: start","job_status":"start","queue_duration_s":0.445774,"scheduling_latency_s":0.445788}2336gitlab # [ 258.292040] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ace4dbd94c78cea4b68781e5","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ace4dbd94c78cea4b68781e5: done: 0.011244 sec","job_status":"done","queue_duration_s":0.445774,"scheduling_latency_s":0.445788,"redis_calls":13,"redis_duration_s":0.003096,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011244,"completed_at":"2026-09-08T04:27:12.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2337gitlab # [ 258.354523] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0c6f8048279292d69c9adc","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0c6f8048279292d69c9adc: start","job_status":"start","queue_duration_s":0.457902,"scheduling_latency_s":0.457915}2338gitlab # [ 258.355297] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0c6f8048279292d69c9adc","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0c6f8048279292d69c9adc: done: 0.010377 sec","job_status":"done","queue_duration_s":0.457902,"scheduling_latency_s":0.457915,"redis_calls":13,"redis_duration_s":0.001702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010377,"completed_at":"2026-09-08T04:27:12.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2339gitlab # [ 258.400345] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec2c3f7832675ed7ce0b1a9a","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec2c3f7832675ed7ce0b1a9a: start","job_status":"start","queue_duration_s":0.469183,"scheduling_latency_s":0.469197}2340gitlab # [ 258.400985] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec2c3f7832675ed7ce0b1a9a","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec2c3f7832675ed7ce0b1a9a: done: 0.010399 sec","job_status":"done","queue_duration_s":0.469183,"scheduling_latency_s":0.469197,"redis_calls":13,"redis_duration_s":0.0018059999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010399,"completed_at":"2026-09-08T04:27:12.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2341gitlab # [ 258.416424] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6e9b1b9fb78065759d6a1fe","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6e9b1b9fb78065759d6a1fe: start","job_status":"start","queue_duration_s":0.480461,"scheduling_latency_s":0.480475}2342gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2343gitlab # [ 258.462090] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6e9b1b9fb78065759d6a1fe","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6e9b1b9fb78065759d6a1fe: done: 0.010297 sec","job_status":"done","queue_duration_s":0.480461,"scheduling_latency_s":0.480475,"redis_calls":13,"redis_duration_s":0.002261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010297,"completed_at":"2026-09-08T04:27:12.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2344gitlab # [ 258.463122] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98907817cf9fc6f868b6495a","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98907817cf9fc6f868b6495a: start","job_status":"start","queue_duration_s":0.491622,"scheduling_latency_s":0.491635}2345gitlab # [ 258.501050] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98907817cf9fc6f868b6495a","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98907817cf9fc6f868b6495a: done: 0.009473 sec","job_status":"done","queue_duration_s":0.491622,"scheduling_latency_s":0.491635,"redis_calls":13,"redis_duration_s":0.001652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009473,"completed_at":"2026-09-08T04:27:12.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2346gitlab # [ 258.504479] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef77ac90b551a34968a0b8f9","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef77ac90b551a34968a0b8f9: start","job_status":"start","queue_duration_s":0.503033,"scheduling_latency_s":0.503065}2347gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.09 seconds)2348gitlab # [ 258.515628] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef77ac90b551a34968a0b8f9","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef77ac90b551a34968a0b8f9: done: 0.012818 sec","job_status":"done","queue_duration_s":0.503033,"scheduling_latency_s":0.503065,"redis_calls":13,"redis_duration_s":0.0015480000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012818,"completed_at":"2026-09-08T04:27:12.751Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2349gitlab # [ 258.556034] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9019342566f61ef78ce6f6a6","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9019342566f61ef78ce6f6a6: start","job_status":"start","queue_duration_s":0.516841,"scheduling_latency_s":0.516859}2350gitlab # [ 258.561037] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9019342566f61ef78ce6f6a6","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9019342566f61ef78ce6f6a6: done: 0.010678 sec","job_status":"done","queue_duration_s":0.516841,"scheduling_latency_s":0.516859,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010678,"completed_at":"2026-09-08T04:27:12.763Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2351gitlab # [ 258.605999] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28327ee8b7c3c05984411a13","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28327ee8b7c3c05984411a13: start","job_status":"start","queue_duration_s":0.528353,"scheduling_latency_s":0.528367}2352gitlab # [ 258.606984] gitaly[932]: time="2026-09-08T04:27:13.803Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMA9ZDMVF2XHN5N2Q3644J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:43.702" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:13.802" grpc.time_ms=0.522 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2353gitlab # [ 258.617352] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 432354gitlab # [ 258.619560] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28327ee8b7c3c05984411a13","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28327ee8b7c3c05984411a13: done: 0.011599 sec","job_status":"done","queue_duration_s":0.528353,"scheduling_latency_s":0.528367,"redis_calls":13,"redis_duration_s":0.001885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011599,"completed_at":"2026-09-08T04:27:12.776Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2355gitlab # [ 258.666032] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07a5665f9fd5c7efda901e95","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07a5665f9fd5c7efda901e95: start","job_status":"start","queue_duration_s":0.54085,"scheduling_latency_s":0.540864}2356gitlab # [ 258.666714] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07a5665f9fd5c7efda901e95","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07a5665f9fd5c7efda901e95: done: 0.010589 sec","job_status":"done","queue_duration_s":0.54085,"scheduling_latency_s":0.540864,"redis_calls":13,"redis_duration_s":0.001797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010589,"completed_at":"2026-09-08T04:27:12.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2357gitlab # [ 258.730518] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30f22fba6e841d33656f7108","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30f22fba6e841d33656f7108: start","job_status":"start","queue_duration_s":0.552353,"scheduling_latency_s":0.552365}2358gitlab # [ 258.731282] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30f22fba6e841d33656f7108","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30f22fba6e841d33656f7108: done: 0.010874 sec","job_status":"done","queue_duration_s":0.552353,"scheduling_latency_s":0.552365,"redis_calls":13,"redis_duration_s":0.001829,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010874,"completed_at":"2026-09-08T04:27:12.799Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2359gitlab # [ 258.775652] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9f71bb1d961be2303e89ecb","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9f71bb1d961be2303e89ecb: start","job_status":"start","queue_duration_s":0.564499,"scheduling_latency_s":0.564512}2360gitlab # [ 258.776444] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9f71bb1d961be2303e89ecb","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9f71bb1d961be2303e89ecb: done: 0.01124 sec","job_status":"done","queue_duration_s":0.564499,"scheduling_latency_s":0.564512,"redis_calls":13,"redis_duration_s":0.001815,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01124,"completed_at":"2026-09-08T04:27:12.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2361gitlab # [ 258.819732] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af612a34b129ab8457466baa","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af612a34b129ab8457466baa: start","job_status":"start","queue_duration_s":0.576892,"scheduling_latency_s":0.576911}2362gitlab # [ 258.820617] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af612a34b129ab8457466baa","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af612a34b129ab8457466baa: done: 0.010029 sec","job_status":"done","queue_duration_s":0.576892,"scheduling_latency_s":0.576911,"redis_calls":13,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010029,"completed_at":"2026-09-08T04:27:12.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2363gitlab # [ 258.837131] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5c61cda80862eba26b7eb50","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5c61cda80862eba26b7eb50: start","job_status":"start","queue_duration_s":0.588252,"scheduling_latency_s":0.588265}2364gitlab # [ 258.880889] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5c61cda80862eba26b7eb50","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5c61cda80862eba26b7eb50: done: 0.009338 sec","job_status":"done","queue_duration_s":0.588252,"scheduling_latency_s":0.588265,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009338,"completed_at":"2026-09-08T04:27:12.833Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2365gitlab # [ 258.883703] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b327fb75e254beef3d2937c0","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b327fb75e254beef3d2937c0: start","job_status":"start","queue_duration_s":0.59841,"scheduling_latency_s":0.598422}2366gitlab # [ 258.895080] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b327fb75e254beef3d2937c0","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b327fb75e254beef3d2937c0: done: 0.010001 sec","job_status":"done","queue_duration_s":0.59841,"scheduling_latency_s":0.598422,"redis_calls":13,"redis_duration_s":0.001421,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010001,"completed_at":"2026-09-08T04:27:12.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2367gitlab # [ 258.962750] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b400e4b71ba406091fb1966","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b400e4b71ba406091fb1966: start","job_status":"start","queue_duration_s":0.609226,"scheduling_latency_s":0.609239}2368gitlab # [ 258.963700] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b400e4b71ba406091fb1966","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b400e4b71ba406091fb1966: done: 0.009174 sec","job_status":"done","queue_duration_s":0.609226,"scheduling_latency_s":0.609239,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009174,"completed_at":"2026-09-08T04:27:12.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2369gitlab # [ 259.010723] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83ea07ed0ececa88a88d98b7","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83ea07ed0ececa88a88d98b7: start","job_status":"start","queue_duration_s":0.619701,"scheduling_latency_s":0.619714}2370gitlab # [ 259.011652] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83ea07ed0ececa88a88d98b7","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83ea07ed0ececa88a88d98b7: done: 0.008915 sec","job_status":"done","queue_duration_s":0.619701,"scheduling_latency_s":0.619714,"redis_calls":13,"redis_duration_s":0.0014479999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008915,"completed_at":"2026-09-08T04:27:12.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2371gitlab # [ 259.059641] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9060db298341364430c749c5","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9060db298341364430c749c5: start","job_status":"start","queue_duration_s":0.629429,"scheduling_latency_s":0.629442}2372gitlab # [ 259.060583] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9060db298341364430c749c5","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9060db298341364430c749c5: done: 0.008905 sec","job_status":"done","queue_duration_s":0.629429,"scheduling_latency_s":0.629442,"redis_calls":13,"redis_duration_s":0.0014500000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008905,"completed_at":"2026-09-08T04:27:12.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2373gitlab # [ 259.070714] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70402ad92c38c5087c15f1f0","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70402ad92c38c5087c15f1f0: start","job_status":"start","queue_duration_s":0.639346,"scheduling_latency_s":0.639359}2374gitlab # [ 259.111715] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70402ad92c38c5087c15f1f0","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70402ad92c38c5087c15f1f0: done: 0.009059 sec","job_status":"done","queue_duration_s":0.639346,"scheduling_latency_s":0.639359,"redis_calls":13,"redis_duration_s":0.001755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009059,"completed_at":"2026-09-08T04:27:12.884Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2375gitlab # [ 259.117060] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b826d028ca37092a48aec676","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b826d028ca37092a48aec676: start","job_status":"start","queue_duration_s":0.649204,"scheduling_latency_s":0.649216}2376gitlab # [ 259.156886] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b826d028ca37092a48aec676","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b826d028ca37092a48aec676: done: 0.009066 sec","job_status":"done","queue_duration_s":0.649204,"scheduling_latency_s":0.649216,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009066,"completed_at":"2026-09-08T04:27:12.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2377gitlab # [ 259.163623] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aedbdb71488ecb4a3f765064","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aedbdb71488ecb4a3f765064: start","job_status":"start","queue_duration_s":0.659365,"scheduling_latency_s":0.659377}2378gitlab # [ 259.209909] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aedbdb71488ecb4a3f765064","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aedbdb71488ecb4a3f765064: done: 0.009062 sec","job_status":"done","queue_duration_s":0.659365,"scheduling_latency_s":0.659377,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009062,"completed_at":"2026-09-08T04:27:12.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2379gitlab # [ 259.210927] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15bd00a84df708e8b811a2ca","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15bd00a84df708e8b811a2ca: start","job_status":"start","queue_duration_s":0.669222,"scheduling_latency_s":0.669235}2380gitlab # [ 259.221578] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15bd00a84df708e8b811a2ca","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15bd00a84df708e8b811a2ca: done: 0.009205 sec","job_status":"done","queue_duration_s":0.669222,"scheduling_latency_s":0.669235,"redis_calls":13,"redis_duration_s":0.001686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009205,"completed_at":"2026-09-08T04:27:12.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2381gitlab # [ 259.292867] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"465c8e7650b7e91e59ce86b3","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-465c8e7650b7e91e59ce86b3: start","job_status":"start","queue_duration_s":0.679472,"scheduling_latency_s":0.679486}2382gitlab # [ 259.293745] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"465c8e7650b7e91e59ce86b3","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-465c8e7650b7e91e59ce86b3: done: 0.008918 sec","job_status":"done","queue_duration_s":0.679472,"scheduling_latency_s":0.679486,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008918,"completed_at":"2026-09-08T04:27:12.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2383gitlab # [ 259.333780] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8edabce19cad34273534f1c5","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8edabce19cad34273534f1c5: start","job_status":"start","queue_duration_s":0.689201,"scheduling_latency_s":0.689214}2384gitlab # [ 259.334684] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8edabce19cad34273534f1c5","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8edabce19cad34273534f1c5: done: 0.009201 sec","job_status":"done","queue_duration_s":0.689201,"scheduling_latency_s":0.689214,"redis_calls":13,"redis_duration_s":0.001427,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009201,"completed_at":"2026-09-08T04:27:12.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2385gitlab # [ 259.350406] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b301e9a3c8d6597059caac","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b301e9a3c8d6597059caac: start","job_status":"start","queue_duration_s":0.700744,"scheduling_latency_s":0.700767}2386gitlab # [ 259.361917] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b301e9a3c8d6597059caac","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b301e9a3c8d6597059caac: done: 0.011679 sec","job_status":"done","queue_duration_s":0.700744,"scheduling_latency_s":0.700767,"redis_calls":13,"redis_duration_s":0.0015079999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011679,"completed_at":"2026-09-08T04:27:12.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2387gitlab # [ 259.405148] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e9a5b2fbedddc77f9abe123","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e9a5b2fbedddc77f9abe123: start","job_status":"start","queue_duration_s":0.713214,"scheduling_latency_s":0.713227}2388gitlab # [ 259.444908] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e9a5b2fbedddc77f9abe123","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e9a5b2fbedddc77f9abe123: done: 0.009149 sec","job_status":"done","queue_duration_s":0.713214,"scheduling_latency_s":0.713227,"redis_calls":13,"redis_duration_s":0.0014579999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009149,"completed_at":"2026-09-08T04:27:12.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2389gitlab # [ 259.445873] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d75af939d71dcfb768842587","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d75af939d71dcfb768842587: start","job_status":"start","queue_duration_s":0.723194,"scheduling_latency_s":0.723208}2390gitlab # [ 259.488823] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d75af939d71dcfb768842587","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d75af939d71dcfb768842587: done: 0.00892 sec","job_status":"done","queue_duration_s":0.723194,"scheduling_latency_s":0.723208,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00892,"completed_at":"2026-09-08T04:27:12.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2391gitlab # [ 259.490129] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cc3f171317da710ee85db96","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cc3f171317da710ee85db96: start","job_status":"start","queue_duration_s":0.732922,"scheduling_latency_s":0.732935}2392gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2393gitlab # [ 259.534760] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cc3f171317da710ee85db96","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cc3f171317da710ee85db96: done: 0.009051 sec","job_status":"done","queue_duration_s":0.732922,"scheduling_latency_s":0.732935,"redis_calls":13,"redis_duration_s":0.0014399999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009051,"completed_at":"2026-09-08T04:27:12.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2394gitlab # [ 259.536604] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1452023876e8040a14e70835","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1452023876e8040a14e70835: start","job_status":"start","queue_duration_s":0.74279,"scheduling_latency_s":0.742803}2395gitlab # [ 259.548169] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1452023876e8040a14e70835","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1452023876e8040a14e70835: done: 0.008946 sec","job_status":"done","queue_duration_s":0.74279,"scheduling_latency_s":0.742803,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008946,"completed_at":"2026-09-08T04:27:12.987Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2396gitlab # [ 259.601765] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:12.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ad515f3e21e7de9f60af7f","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ad515f3e21e7de9f60af7f: start","job_status":"start","queue_duration_s":0.752703,"scheduling_latency_s":0.752715}2397gitlab # [ 259.602639] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ad515f3e21e7de9f60af7f","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ad515f3e21e7de9f60af7f: done: 0.011736 sec","job_status":"done","queue_duration_s":0.752703,"scheduling_latency_s":0.752715,"redis_calls":13,"redis_duration_s":0.00182,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011736,"completed_at":"2026-09-08T04:27:13.000Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2398gitlab # [ 259.628793] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b36d6b63083621217cb41a","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b36d6b63083621217cb41a: start","job_status":"start","queue_duration_s":0.765288,"scheduling_latency_s":0.765302}2399gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.15 seconds)2400gitlab # [ 259.672987] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b36d6b63083621217cb41a","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b36d6b63083621217cb41a: done: 0.009023 sec","job_status":"done","queue_duration_s":0.765288,"scheduling_latency_s":0.765302,"redis_calls":13,"redis_duration_s":0.0014839999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009023,"completed_at":"2026-09-08T04:27:13.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2401gitlab # [ 259.674558] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d6e30f0c3a21aaf6473c41c","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d6e30f0c3a21aaf6473c41c: start","job_status":"start","queue_duration_s":0.775332,"scheduling_latency_s":0.775345}2402gitlab # [ 259.719374] gitaly[932]: time="2026-09-08T04:27:14.928Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMAB1VTFSC6FHQRBP6N3A0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:44.427" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:14.927" grpc.time_ms=0.88 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2403gitlab # [ 259.720263] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 862404gitlab # [ 259.720729] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d6e30f0c3a21aaf6473c41c","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d6e30f0c3a21aaf6473c41c: done: 0.009159 sec","job_status":"done","queue_duration_s":0.775332,"scheduling_latency_s":0.775345,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009159,"completed_at":"2026-09-08T04:27:13.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2405gitlab # [ 259.761084] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d00c310663a449e760b1f71","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d00c310663a449e760b1f71: start","job_status":"start","queue_duration_s":0.785314,"scheduling_latency_s":0.785327}2406gitlab # [ 259.761780] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d00c310663a449e760b1f71","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d00c310663a449e760b1f71: done: 0.0094 sec","job_status":"done","queue_duration_s":0.785314,"scheduling_latency_s":0.785327,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0094,"completed_at":"2026-09-08T04:27:13.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2407gitlab # [ 259.807603] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"949d540fdd3f35254a0c7aa8","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-949d540fdd3f35254a0c7aa8: start","job_status":"start","queue_duration_s":0.795542,"scheduling_latency_s":0.795555}2408gitlab # [ 259.808529] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"949d540fdd3f35254a0c7aa8","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-949d540fdd3f35254a0c7aa8: done: 0.008925 sec","job_status":"done","queue_duration_s":0.795542,"scheduling_latency_s":0.795555,"redis_calls":13,"redis_duration_s":0.001462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008925,"completed_at":"2026-09-08T04:27:13.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2409gitlab # [ 259.858706] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"308644a48781edcb16e3e15d","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-308644a48781edcb16e3e15d: start","job_status":"start","queue_duration_s":0.805272,"scheduling_latency_s":0.805284}2410gitlab # [ 259.859580] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"308644a48781edcb16e3e15d","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-308644a48781edcb16e3e15d: done: 0.008829 sec","job_status":"done","queue_duration_s":0.805272,"scheduling_latency_s":0.805284,"redis_calls":13,"redis_duration_s":0.001449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008829,"completed_at":"2026-09-08T04:27:13.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2411gitlab # [ 259.868592] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d9e743ccfbc974a45924fb3","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d9e743ccfbc974a45924fb3: start","job_status":"start","queue_duration_s":0.814908,"scheduling_latency_s":0.814921}2412gitlab # [ 259.911494] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d9e743ccfbc974a45924fb3","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d9e743ccfbc974a45924fb3: done: 0.009127 sec","job_status":"done","queue_duration_s":0.814908,"scheduling_latency_s":0.814921,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009127,"completed_at":"2026-09-08T04:27:13.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2413gitlab # [ 259.913622] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"398f6bb8efefabc7015e4aea","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-398f6bb8efefabc7015e4aea: start","job_status":"start","queue_duration_s":0.824858,"scheduling_latency_s":0.824871}2414gitlab # [ 259.924915] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:13.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"398f6bb8efefabc7015e4aea","created_at":"2026-09-08T04:27:12.190Z","trace_propagation_headers":{"sentry-trace":"c56b3798e6ad435b9d4721c11ff2fedc-f8f8cae2854f4292","baggage":"sentry-trace_id=c56b3798e6ad435b9d4721c11ff2fedc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3cf7e0dc2bc3729a7cc4f8c6e162b8b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-08T04:27:12.236Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-398f6bb8efefabc7015e4aea: done: 0.00892 sec","job_status":"done","queue_duration_s":0.824858,"scheduling_latency_s":0.824871,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00892,"completed_at":"2026-09-08T04:27:13.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2415gitlab # [ 260.103596] gitaly[932]: time="2026-09-08T04:27:15.363Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:45.361" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:15.362" grpc.time_ms=1.019 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2416gitlab # [ 260.141604] gitaly[932]: time="2026-09-08T04:27:15.401Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:25.271" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.391" grpc.time_ms=9.614 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2417gitlab # [ 260.186243] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.446Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c09a60add64b16fc38f5d009","created_at":"2026-09-08T04:27:15.441Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-85aded3029fb4b2f","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.444Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c09a60add64b16fc38f5d009: start","job_status":"start","queue_duration_s":0.001325,"scheduling_latency_s":0.001356}2418gitlab # [ 260.230049] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1982419gitlab # [ 260.234533] nixbot[2663]: {"ts": "2026-09-08T04:27:15.494Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 \"HTTP/1.1 201 Created\""}2420gitlab # [ 260.275331] gitaly[932]: time="2026-09-08T04:27:15.535Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABNARJ1Q6XQP6V95QQN7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:45.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:15.534" grpc.time_ms=0.443 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2421gitlab # [ 260.301629] gitaly[932]: time="2026-09-08T04:27:15.561Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABNARJ1Q6XQP6V95QQN7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:25.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.556" grpc.time_ms=5.174 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2422gitlab # [ 260.322032] gitaly[932]: time="2026-09-08T04:27:15.582Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:15.581" grpc.time_ms=0.982 pid=932 span.kind=server system=grpc2423gitlab # [ 260.335088] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 962424gitlab # [ 260.337754] nixbot[2663]: {"ts": "2026-09-08T04:27:15.596Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 \"HTTP/1.1 201 Created\""}2425gitlab # [ 260.349513] gitaly[932]: time="2026-09-08T04:27:15.609Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:15.609" grpc.time_ms=0.544 pid=932 span.kind=server system=grpc2426gitlab # [ 260.375301] gitaly[932]: time="2026-09-08T04:27:15.635Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-08T04:27:15.634" grpc.time_ms=0.355 pid=932 span.kind=server system=grpc2427gitlab # [ 260.387602] gitaly[932]: time="2026-09-08T04:27:15.647Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:45.527" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.528" operations="[repack_refs]" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2428gitlab # [ 260.388932] gitaly[932]: time="2026-09-08T04:27:15.649Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABRQRD0XK0J9JRCYSR5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:45.548" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:15.648" grpc.time_ms=0.521 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2429gitlab # [ 260.393707] gitaly[932]: time="2026-09-08T04:27:15.653Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV diskcache=653ce054-c8e2-4d13-8a86-517606dd31a7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:45.527" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.528" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2430gitlab # [ 260.394625] gitaly[932]: time="2026-09-08T04:27:15.654Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=85 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7159 command.oublock=0 command.real_time_ms=118 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:45.527" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.528" grpc.time_ms=125.481 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2431gitlab # [ 260.418471] gitaly[932]: time="2026-09-08T04:27:15.678Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABRQRD0XK0J9JRCYSR5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-08T04:27:25.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.673" grpc.time_ms=5.415 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2432gitlab # [ 260.443851] gitaly[932]: time="2026-09-08T04:27:15.703Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:45.527" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.528" pid=932 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-08 04:26:05.658897827 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2433gitlab # [ 260.455089] gitaly[932]: time="2026-09-08T04:27:15.704Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-08T04:27:45.702" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:15.702" grpc.time_ms=1.092 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2434gitlab # [ 260.467472] gitlab-workhorse[833]: gitlab 2001:db8:1::1 - - [2026/09/08:04:27:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1072435gitlab # [ 260.467887] nixbot[2663]: {"ts": "2026-09-08T04:27:15.715Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9b05cebdf8aa414871b7f2c72bf1e8b712287066 \"HTTP/1.1 201 Created\""}2436gitlab # [ 260.468532] gitaly[932]: time="2026-09-08T04:27:15.716Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:45.527" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.528" pid=932 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2437gitlab # [ 260.469187] gitaly[932]: time="2026-09-08T04:27:15.716Z" level=info msg="optimized repository" clean_stale_data_time=12.647417ms clean_work_trees_time="68.723µs" commit_graph_time="3.911µs" correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=932 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.749µs" storage_name=default system=housekeeping total_time=12.823417ms2438gitlab # [ 260.469667] gitaly[932]: time="2026-09-08T04:27:15.716Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-08T04:27:45.527" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-08T04:27:15.528" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2439gitlab # [ 260.509261] gitaly[932]: time="2026-09-08T04:27:15.769Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1ZMABEV7FZ2YXMQF44VXYBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-08T04:27:45.768" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-08T04:27:15.768" grpc.time_ms=0.509 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2440gitlab # [ 260.529551] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.789Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"02a882005eda11a840c067f3","created_at":"2026-09-08T04:27:15.787Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"c09a60add64b16fc38f5d009","job_size_bytes":3,"pid":2540,"job_status":"deduplicated","message":"PipelineProcessWorker JID-02a882005eda11a840c067f3: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2441gitlab # [ 260.544032] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.790Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c09a60add64b16fc38f5d009","created_at":"2026-09-08T04:27:15.441Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-85aded3029fb4b2f","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.444Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2540,"job_status":"rescheduled","message":"PipelineProcessWorker JID-c09a60add64b16fc38f5d009: rescheduled"}2442gitlab # [ 260.544311] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.795Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c09a60add64b16fc38f5d009","created_at":"2026-09-08T04:27:15.441Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-85aded3029fb4b2f","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.444Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c09a60add64b16fc38f5d009: done: 0.349438 sec","job_status":"done","queue_duration_s":0.001325,"scheduling_latency_s":0.001356,"gitaly_calls":3,"gitaly_duration_s":0.155131,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011576,"redis_read_bytes":239,"redis_write_bytes":21783,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000698,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001204,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002242,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14468,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.004613,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000156,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000352,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001613,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.115255,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.299775809,"duration_s":0.349438,"completed_at":"2026-09-08T04:27:15.795Z","load_balancing_strategy":"primary","db_duration_s":0.041053,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2443gitlab # [ 260.586058] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.796Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"da7d2f80fc6b0d6897cf8504","created_at":"2026-09-08T04:27:15.522Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.523Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-da7d2f80fc6b0d6897cf8504: start","job_status":"start","queue_duration_s":0.2725,"scheduling_latency_s":0.272516}2444gitlab # [ 260.586652] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.828Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"da7d2f80fc6b0d6897cf8504","created_at":"2026-09-08T04:27:15.522Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.523Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-da7d2f80fc6b0d6897cf8504: done: 0.031857 sec","job_status":"done","queue_duration_s":0.2725,"scheduling_latency_s":0.272516,"redis_calls":7,"redis_duration_s":0.0011459999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031857,"completed_at":"2026-09-08T04:27:15.828Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006338,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2445gitlab # [ 260.633082] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.829Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ef7068226d3d48c5daf45217","created_at":"2026-09-08T04:27:15.660Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.662Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ef7068226d3d48c5daf45217: start","job_status":"start","queue_duration_s":0.167618,"scheduling_latency_s":0.167639}2446gitlab # [ 260.633739] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.848Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ef7068226d3d48c5daf45217","created_at":"2026-09-08T04:27:15.660Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.662Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ef7068226d3d48c5daf45217: done: 0.018679 sec","job_status":"done","queue_duration_s":0.167618,"scheduling_latency_s":0.167639,"redis_calls":7,"redis_duration_s":0.0012360000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018679,"completed_at":"2026-09-08T04:27:15.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2447gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses2448gitlab # [ 260.673184] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"39ff4696897247db380d903c","created_at":"2026-09-08T04:27:15.668Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.673Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-39ff4696897247db380d903c: start","job_status":"start","queue_duration_s":0.175394,"scheduling_latency_s":0.175409}2449gitlab # [ 260.713221] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.865Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"39ff4696897247db380d903c","created_at":"2026-09-08T04:27:15.668Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.673Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-39ff4696897247db380d903c: done: 0.016633 sec","job_status":"done","queue_duration_s":0.175394,"scheduling_latency_s":0.175409,"redis_calls":7,"redis_duration_s":0.000912,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016633,"completed_at":"2026-09-08T04:27:15.865Z","load_balancing_strategy":"primary","db_duration_s":0.002594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2450gitlab # [ 260.717693] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.866Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"a7eed0ff7815e971432b222b","created_at":"2026-09-08T04:27:15.679Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.680Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a7eed0ff7815e971432b222b: start","job_status":"start","queue_duration_s":0.186179,"scheduling_latency_s":0.186193}2451gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/9b05cebdf8aa414871b7f2c72bf1e8b712287066/statuses, in 0.10 seconds)2452(finished: subtest: eval and build post commit statuses, in 8.30 seconds)2453(finished: run the VM test script, in 316.45 seconds)2454gitlab # [ 260.764126] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"a7eed0ff7815e971432b222b","created_at":"2026-09-08T04:27:15.679Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.680Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a7eed0ff7815e971432b222b: done: 0.01821 sec","job_status":"done","queue_duration_s":0.186179,"scheduling_latency_s":0.186193,"redis_calls":10,"redis_duration_s":0.001238,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000355,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01821,"completed_at":"2026-09-08T04:27:15.885Z","load_balancing_strategy":"primary","db_duration_s":0.002952,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2455gitlab # [ 260.769306] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a63fd188b091a178f5112cf2","created_at":"2026-09-08T04:27:15.681Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.681Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a63fd188b091a178f5112cf2: start","job_status":"start","queue_duration_s":0.203964,"scheduling_latency_s":0.203978}2456test script finished in 316.50s2457cleanup2458kill QemuMachine (pid 46)2459gitlab # [ 260.814856] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-09-08T04:27:15.900Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a63fd188b091a178f5112cf2","created_at":"2026-09-08T04:27:15.681Z","trace_propagation_headers":{"sentry-trace":"9fe6d9961be144cf94103a1ed36cd865-910d6c1e21064c54","baggage":"sentry-trace_id=9fe6d9961be144cf94103a1ed36cd865,sentry-release=Unknown"},"correlation_id":"01M1ZMABEV7FZ2YXMQF44VXYBV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-08T04:27:15.681Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a63fd188b091a178f5112cf2: done: 0.014915 sec","job_status":"done","queue_duration_s":0.203964,"scheduling_latency_s":0.203978,"redis_calls":7,"redis_duration_s":0.000858,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_coqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2460gitlab # u2461(finished: cleanup, in 1.68 seconds)2462additionally exposed symbols:2463 gitlab,2464 vlan1,2465 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2466[]2467[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-08T04:26:15.981Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2468[]2469[]2470[]2471[]2472[]2473[]2474[]2475[]2476[]2477[]2478[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-08T04:27:07.022840Z', 'started_at': '2026-09-08T04:27:07.519590Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2479[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2480[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2481[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2482[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2483[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2484[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2485[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2486[{'id': 5, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 7s (1 warning)', 'created_at': '2026-09-08T04:27:07.472Z', 'started_at': None, 'finished_at': '2026-09-08T04:27:15.431Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '9b05cebdf8aa414871b7f2c72bf1e8b712287066', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-08T04:27:15.575Z', 'started_at': None, 'finished_at': '2026-09-08T04:27:15.693Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]