vm-test-run-nixbot-gitlab
x86_64-linux.nixbot-gitlab
· build #120
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 13)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.pqSi5KVAEB', fmt=raw size=107374182412(finished: start all VMs, in 35.14 seconds)13gitlab # mke2fs 1.47.3 (8-Jul-2025)14gitlab: waiting for unit setup-gitlab.service15gitlab # Discarding device blocks: 0/262144 done16gitlab: waiting for the VM to finish booting17gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes18gitlab # Filesystem UUID: 80738cfb-64cd-4cd5-bb65-4e8e7f523a6b19gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... ok45gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 202646gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/15f597533s2qv6mxxjladp3li5g5zvjg-nixos-system-gitlab-test/init regInfo=/nix/store/k6k50mqyg06zcznzgq5yid6bk6mjri5d-closure-info/registration console=ttyS0,115200n8 console=tty047gitlab # [ 0.000000] BIOS-provided physical RAM map:48gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable49gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved51gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved55gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable56gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved57gitlab # [ 0.000000] NX (Execute Disable) protection: active58gitlab # [ 0.000000] APIC: Static calls initialized59gitlab # [ 0.000000] SMBIOS 2.8 present.60gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201461gitlab # [ 0.000000] DMI: Memory slots populated: 1/162gitlab # [ 0.000000] Hypervisor detected: KVM63gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000064gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0065gitlab # [ 0.000000] kvm-clock: using sched offset of 507489241 cycles66gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns67gitlab # [ 0.000005] tsc: Detected 2400.018 MHz processor68gitlab # [ 0.000806] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000069gitlab # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs70gitlab # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT71gitlab # [ 0.000901] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000072gitlab # [ 0.002752] found SMP MP-table at [mem 0x000f5470-0x000f547f]73gitlab # [ 0.002764] Using GB pages for direct mapping74gitlab # [ 0.002824] RAMDISK: [mem 0xbe4e0000-0xbffcffff]75gitlab # [ 0.002832] ACPI: Early table checksum verification disabled76gitlab # [ 0.002835] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )77gitlab # [ 0.002839] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002843] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002850] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)80gitlab # [ 0.002852] ACPI: FACS 0x00000000BFFE0000 00004081gitlab # [ 0.002853] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002855] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002856] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)84gitlab # [ 0.002858] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]85gitlab # [ 0.002859] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]86gitlab # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]87gitlab # [ 0.002860] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]88gitlab # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]89gitlab # [ 0.002861] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]90gitlab # [ 0.003355] No NUMA configuration found91gitlab # [ 0.003357] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]92gitlab # [ 0.003360] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]93gitlab # [ 0.005824] Zone ranges:94gitlab # [ 0.005825] DMA [mem 0x0000000000001000-0x0000000000ffffff]95gitlab # [ 0.005827] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]96gitlab # [ 0.005828] Normal [mem 0x0000000100000000-0x00000001bfffffff]97gitlab # [ 0.005830] Device empty98gitlab # [ 0.005830] Movable zone start for each node99gitlab # [ 0.005831] Early memory node ranges100gitlab # [ 0.005831] node 0: [mem 0x0000000000001000-0x000000000009efff]101gitlab # [ 0.005832] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]102gitlab # [ 0.005833] node 0: [mem 0x0000000100000000-0x00000001bfffffff]103gitlab # [ 0.005834] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]104gitlab # [ 0.005854] On node 0, zone DMA: 1 pages in unavailable ranges105gitlab # [ 0.006144] On node 0, zone DMA: 97 pages in unavailable ranges106gitlab # [ 0.116704] On node 0, zone Normal: 40 pages in unavailable ranges107gitlab # [ 0.117719] ACPI: PM-Timer IO Port: 0x608108gitlab # [ 0.117737] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])109gitlab # [ 0.117777] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23110gitlab # [ 0.117780] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)111gitlab # [ 0.117782] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)112gitlab # [ 0.117783] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)113gitlab # [ 0.117784] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)114gitlab # [ 0.117785] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)115gitlab # [ 0.117787] ACPI: Using ACPI (MADT) for SMP configuration information116gitlab # [ 0.117789] ACPI: HPET id: 0x8086a201 base: 0xfed00000117gitlab # [ 0.117795] TSC deadline timer available118gitlab # [ 0.117799] CPU topo: Max. logical packages: 1119gitlab # [ 0.117800] CPU topo: Max. logical dies: 1120gitlab # [ 0.117801] CPU topo: Max. dies per package: 1121gitlab # [ 0.117804] CPU topo: Max. threads per core: 1122gitlab # [ 0.117804] CPU topo: Num. cores per package: 4123gitlab # [ 0.117805] CPU topo: Num. threads per package: 4124gitlab # [ 0.117805] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs125gitlab # [ 0.117833] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()126gitlab # [ 0.117852] kvm-guest: KVM setup pv remote TLB flush127gitlab # [ 0.117855] kvm-guest: setup PV sched yield128gitlab # [ 0.117875] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]129gitlab # [ 0.117876] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]130gitlab # [ 0.117877] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]131gitlab # [ 0.117879] [mem 0xc0000000-0xfeffbfff] available for PCI devices132gitlab # [ 0.117881] Booting paravirtualized kernel on KVM133gitlab # [ 0.117885] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns134gitlab # [ 0.122371] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1135gitlab # [ 0.124775] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288136gitlab # [ 0.124827] kvm-guest: PV spinlocks enabled137gitlab # [ 0.124829] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)138gitlab # [ 0.124832] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/15f597533s2qv6mxxjladp3li5g5zvjg-nixos-system-gitlab-test/init regInfo=/nix/store/k6k50mqyg06zcznzgq5yid6bk6mjri5d-closure-info/registration console=ttyS0,115200n8 console=tty0139gitlab # [ 0.124932] Unknown kernel command line parameters "regInfo=/nix/store/k6k50mqyg06zcznzgq5yid6bk6mjri5d-closure-info/registration", will be passed to user space.140gitlab # [ 0.124944] random: crng init done141gitlab # [ 0.124945] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes142gitlab # [ 0.133897] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)143gitlab # [ 0.138297] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)144gitlab # [ 0.138389] software IO TLB: area num 4.145gitlab # [ 0.214618] Fallback order for Node 0: 0146gitlab # [ 0.214628] Built 1 zonelists, mobility grouping on. Total pages: 1572726147gitlab # [ 0.214630] Policy zone: Normal148gitlab # [ 0.217590] mem auto-init: stack:all(zero), heap alloc:on, heap free:off149gitlab # [ 0.227478] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1150gitlab # [ 0.241366] allocated 12582912 bytes of page_ext151gitlab # [ 0.251502] ftrace: allocating 48565 entries in 192 pages152gitlab # [ 0.251504] ftrace: allocated 192 pages with 2 groups153gitlab # [ 0.252388] Dynamic Preempt: lazy154gitlab # [ 0.252543] rcu: Preemptible hierarchical RCU implementation.155gitlab # [ 0.252544] rcu: RCU event tracing is enabled.156gitlab # [ 0.252545] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.157gitlab # [ 0.252546] Trampoline variant of Tasks RCU enabled.158gitlab # [ 0.252547] Rude variant of Tasks RCU enabled.159gitlab # [ 0.252547] Tracing variant of Tasks RCU enabled.160gitlab # [ 0.252548] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.161gitlab # [ 0.252548] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4162gitlab # [ 0.252568] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.252570] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.252571] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.165gitlab # [ 0.257163] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16166gitlab # [ 0.257453] rcu: srcu_init: Setting srcu_struct sizes based on contention.167gitlab # [ 0.257579] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.264799] Console: colour VGA+ 80x25169gitlab # [ 0.264802] printk: legacy console [tty0] enabled170gitlab # [ 0.308346] printk: legacy console [ttyS0] enabled171gitlab # [ 0.505161] ACPI: Core revision 20250807172gitlab # [ 0.506730] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.509575] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.511317] x2apic enabled175gitlab # [ 0.512557] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.514257] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.516444] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.521344] kvm-guest: setup PV IPIs179gitlab # [ 0.523854] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.525706] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229848b50c4, max_idle_ns: 440795322125 ns181gitlab # [ 0.528799] Calibrating delay loop (skipped) preset value.. 4800.03 BogoMIPS (lpj=2400018)182gitlab # [ 0.529915] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.531943] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.533798] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.535802] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.537798] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.538799] Transient Scheduler Attacks: Mitigation: Clear CPU buffers188gitlab # [ 0.540797] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.542798] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.543798] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.544803] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier192gitlab # [ 0.546798] active return thunk: srso_alias_return_thunk193gitlab # [ 0.547826] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'194gitlab # [ 0.548798] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'195gitlab # [ 0.549797] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'196gitlab # [ 0.550798] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'197gitlab # [ 0.551797] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'198gitlab # [ 0.552797] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'199gitlab # [ 0.553798] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'200gitlab # [ 0.555798] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'201gitlab # [ 0.556798] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'202gitlab # [ 0.558798] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256203gitlab # [ 0.559797] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64204gitlab # [ 0.560798] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512205gitlab # [ 0.562798] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024206gitlab # [ 0.564798] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8207gitlab # [ 0.566798] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16208gitlab # [ 0.567798] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24209gitlab # [ 0.568798] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.210gitlab # [ 0.599022] Freeing SMP alternatives memory: 44K211gitlab # [ 0.599801] pid_max: default: 32768 minimum: 301212gitlab # [ 0.600916] LSM: initializing lsm=capability,landlock,yama,bpf,ima213gitlab # [ 0.601913] landlock: Up and running.214gitlab # [ 0.602798] Yama: becoming mindful.215gitlab # [ 0.604011] LSM support for eBPF active216gitlab # [ 0.606020] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)217gitlab # [ 0.606945] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)218gitlab # [ 0.610096] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)219gitlab # [ 0.611391] Performance Events: Fam17h+ core perfctr, AMD PMU driver.220gitlab # [ 0.611808] ... version: 2221gitlab # [ 0.612799] ... bit width: 48222gitlab # [ 0.613799] ... generic counters: 6223gitlab # [ 0.614799] ... generic bitmap: 000000000000003f224gitlab # [ 0.615800] ... fixed-purpose counters: 0225gitlab # [ 0.616799] ... fixed-purpose bitmap: 0000000000000000226gitlab # [ 0.617800] ... value mask: 0000ffffffffffff227gitlab # [ 0.618799] ... max period: 00007fffffffffff228gitlab # [ 0.619799] ... global_ctrl mask: 000000000000003f229gitlab # [ 0.620910] signal: max sigframe size: 3376230gitlab # [ 0.621888] rcu: Hierarchical SRCU implementation.231gitlab # [ 0.622804] rcu: Max phase no-delay instances is 400.232gitlab # [ 0.623997] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level233gitlab # [ 0.629529] smp: Bringing up secondary CPUs ...234gitlab # [ 0.631211] smpboot: x86: Booting SMP configuration:235gitlab # [ 0.631810] .... node #0, CPUs: #1 #2 #3236gitlab # [ 0.635893] smp: Brought up 1 node, 4 CPUs237gitlab # [ 0.638803] smpboot: Total of 4 processors activated (19200.14 BogoMIPS)238gitlab # [ 0.640207] Memory: 6013180K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved)239gitlab # [ 0.642797] devtmpfs: initialized240gitlab # [ 0.644028] x86/mm: Memory block size: 128MB241gitlab # [ 0.647812] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns242gitlab # [ 0.648853] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)243gitlab # [ 0.649936] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).244gitlab # [ 0.650926] pinctrl core: initialized pinctrl subsystem245gitlab # [ 0.652174] PM: RTC time: 06:48:49, date: 2026-07-09246gitlab # [ 0.655695] NET: Registered PF_NETLINK/PF_ROUTE protocol family247gitlab # [ 0.658128] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations248gitlab # [ 0.658872] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations249gitlab # [ 0.660905] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations250gitlab # [ 0.661812] audit: initializing netlink subsys (disabled)251gitlab # [ 0.663001] audit: type=2000 audit(1783579729.527:1): state=initialized audit_enabled=0 res=1252gitlab # [ 0.663081] thermal_sys: Registered thermal governor 'fair_share'253gitlab # [ 0.664800] thermal_sys: Registered thermal governor 'bang_bang'254gitlab # [ 0.665800] thermal_sys: Registered thermal governor 'step_wise'255gitlab # [ 0.666800] thermal_sys: Registered thermal governor 'user_space'256gitlab # [ 0.667799] thermal_sys: Registered thermal governor 'power_allocator'257gitlab # [ 0.668818] cpuidle: using governor menu258gitlab # [ 0.672125] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5259gitlab # [ 0.674117] PCI: Using configuration type 1 for base access260gitlab # [ 0.674800] PCI: Using configuration type 1 for extended access261gitlab # [ 0.676008] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.262gitlab # [ 0.678848] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages263gitlab # [ 0.679800] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page264gitlab # [ 0.680800] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages265gitlab # [ 0.681799] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page266gitlab # [ 0.686605] ACPI: Added _OSI(Module Device)267gitlab # [ 0.689809] ACPI: Added _OSI(Processor Device)268gitlab # [ 0.693811] ACPI: Added _OSI(Processor Aggregator Device)269gitlab # [ 0.702923] ACPI: 1 ACPI AML tables successfully acquired and loaded270gitlab # [ 0.712066] ACPI: Interpreter enabled271gitlab # [ 0.714896] ACPI: PM: (supports S0 S3 S4 S5)272gitlab # [ 0.717821] ACPI: Using IOAPIC for interrupt routing273gitlab # [ 0.722139] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug274gitlab # [ 0.728809] PCI: Using E820 reservations for host bridge windows275gitlab # [ 0.733457] ACPI: Enabled 2 GPEs in block 00 to 0F276gitlab # [ 0.750730] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])277gitlab # [ 0.753816] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]278gitlab # [ 0.760062] acpiphp: Slot [3] registered279gitlab # [ 0.762919] acpiphp: Slot [4] registered280gitlab # [ 0.765920] acpiphp: Slot [5] registered281gitlab # [ 0.767966] acpiphp: Slot [6] registered282gitlab # [ 0.770910] acpiphp: Slot [7] registered283gitlab # [ 0.772863] acpiphp: Slot [8] registered284gitlab # [ 0.774861] acpiphp: Slot [9] registered285gitlab # [ 0.775876] acpiphp: Slot [10] registered286gitlab # [ 0.776877] acpiphp: Slot [11] registered287gitlab # [ 0.777865] acpiphp: Slot [12] registered288gitlab # [ 0.780104] acpiphp: Slot [13] registered289gitlab # [ 0.781838] acpiphp: Slot [14] registered290gitlab # [ 0.782840] acpiphp: Slot [15] registered291gitlab # [ 0.783849] acpiphp: Slot [16] registered292gitlab # [ 0.786908] acpiphp: Slot [17] registered293gitlab # [ 0.789893] acpiphp: Slot [18] registered294gitlab # [ 0.792890] acpiphp: Slot [19] registered295gitlab # [ 0.795928] acpiphp: Slot [20] registered296gitlab # [ 0.799893] acpiphp: Slot [21] registered297gitlab # [ 0.802892] acpiphp: Slot [22] registered298gitlab # [ 0.805891] acpiphp: Slot [23] registered299gitlab # [ 0.808945] acpiphp: Slot [24] registered300gitlab # [ 0.811893] acpiphp: Slot [25] registered301gitlab # [ 0.814894] acpiphp: Slot [26] registered302gitlab # [ 0.817872] acpiphp: Slot [27] registered303gitlab # [ 0.819899] acpiphp: Slot [28] registered304gitlab # [ 0.822941] acpiphp: Slot [29] registered305gitlab # [ 0.824882] acpiphp: Slot [30] registered306gitlab # [ 0.827905] acpiphp: Slot [31] registered307gitlab # [ 0.829982] PCI host bridge to bus 0000:00308gitlab # [ 0.832821] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]309gitlab # [ 0.835802] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]310gitlab # [ 0.837803] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]311gitlab # [ 0.839801] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]312gitlab # [ 0.841810] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]313gitlab # [ 0.843828] pci_bus 0000:00: root bus resource [bus 00-ff]314gitlab # [ 0.846324] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint315gitlab # [ 0.851324] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint316gitlab # [ 0.860133] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint317gitlab # [ 0.870966] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]318gitlab # [ 0.874986] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk319gitlab # [ 0.879808] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk320gitlab # [ 0.884808] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk321gitlab # [ 0.889809] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk322gitlab # [ 0.894591] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint323gitlab # [ 0.904657] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]324gitlab # [ 0.909654] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint325gitlab # [ 0.916305] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI326gitlab # [ 0.921847] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB327gitlab # [ 0.926989] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs328gitlab # [ 0.932924] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint329gitlab # [ 0.955910] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]330gitlab # [ 0.958827] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]331gitlab # [ 0.960861] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]332gitlab # [ 0.963064] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]333gitlab # [ 0.976819] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 13671 usecs334gitlab # [ 0.979995] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint335gitlab # [ 0.991851] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]336gitlab # [ 0.995846] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]337gitlab # [ 0.999938] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]338gitlab # [ 1.005854] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]339gitlab # [ 1.016274] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint340gitlab # [ 1.029843] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]341gitlab # [ 1.032834] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]342gitlab # [ 1.036908] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]343gitlab # [ 1.045112] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint344gitlab # [ 1.056838] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]345gitlab # [ 1.059843] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]346gitlab # [ 1.064938] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]347gitlab # [ 1.076401] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint348gitlab # [ 1.089856] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]349gitlab # [ 1.094234] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]350gitlab # [ 1.097938] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]351gitlab # [ 1.107114] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint352gitlab # [ 1.119853] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]353gitlab # [ 1.121818] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]354gitlab # [ 1.123852] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]355gitlab # [ 1.128044] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint356gitlab # [ 1.135137] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]357gitlab # [ 1.136814] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]358gitlab # [ 1.138852] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]359gitlab # [ 1.142888] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint360gitlab # [ 1.147844] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]361gitlab # [ 1.150815] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]362gitlab # [ 1.152860] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]363gitlab # [ 1.154814] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]364gitlab # [ 1.157850] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint365gitlab # [ 1.163855] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]366gitlab # [ 1.165852] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]367gitlab # [ 1.169955] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint368gitlab # [ 1.175852] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]369gitlab # [ 1.176814] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]370gitlab # [ 1.178853] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]371gitlab # [ 1.185424] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint372gitlab # [ 1.197844] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]373gitlab # [ 1.201845] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]374gitlab # [ 1.205938] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]375gitlab # [ 1.229684] ACPI: PCI: Interrupt link LNKA configured for IRQ 10376gitlab # [ 1.234235] ACPI: PCI: Interrupt link LNKB configured for IRQ 10377gitlab # [ 1.238276] ACPI: PCI: Interrupt link LNKC configured for IRQ 11378gitlab # [ 1.241178] ACPI: PCI: Interrupt link LNKD configured for IRQ 11379gitlab # [ 1.245021] ACPI: PCI: Interrupt link LNKS configured for IRQ 9380gitlab # [ 1.252231] iommu: Default domain type: Translated381gitlab # [ 1.254804] iommu: DMA domain TLB invalidation policy: lazy mode382gitlab # [ 1.259287] ACPI: bus type USB registered383gitlab # [ 1.261967] usbcore: registered new interface driver usbfs384gitlab # [ 1.265990] usbcore: registered new interface driver hub385gitlab # [ 1.269876] usbcore: registered new device driver usb386gitlab # [ 1.277069] NetLabel: Initializing387gitlab # [ 1.279807] NetLabel: domain hash size = 128388gitlab # [ 1.282806] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO389gitlab # [ 1.287039] NetLabel: unlabeled traffic allowed by default390gitlab # [ 1.290841] PCI: Using ACPI for IRQ routing391gitlab # [ 1.296014] pci 0000:00:02.0: vgaarb: setting as boot VGA device392gitlab # [ 1.296794] pci 0000:00:02.0: vgaarb: bridge control possible393gitlab # [ 1.296794] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none394gitlab # [ 1.307812] vgaarb: loaded395gitlab # [ 1.309246] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0396gitlab # [ 1.311799] hpet0: 3 comparators, 64-bit 100.000000 MHz counter397gitlab # [ 1.317073] clocksource: Switched to clocksource kvm-clock398gitlab # [ 1.323401] VFS: Disk quotas dquot_6.6.0399gitlab # [ 1.326543] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)400gitlab # [ 1.332452] pnp: PnP ACPI init401gitlab # [ 1.336696] pnp: PnP ACPI: found 6 devices402gitlab # [ 1.352258] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns403gitlab # [ 1.358761] clocksource: Switched to clocksource acpi_pm404gitlab # [ 1.363291] NET: Registered PF_INET protocol family405gitlab # [ 1.368996] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)406gitlab # [ 1.395504] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)407gitlab # [ 1.402251] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)408gitlab # [ 1.408518] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)409gitlab # [ 1.420574] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)410gitlab # [ 1.426528] TCP: Hash tables configured (established 65536 bind 65536)411gitlab # [ 1.432658] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)412gitlab # [ 1.439208] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)413gitlab # [ 1.445115] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)414gitlab # [ 1.450559] NET: Registered PF_UNIX/PF_LOCAL protocol family415gitlab # [ 1.454011] NET: Registered PF_XDP protocol family416gitlab # [ 1.457104] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]417gitlab # [ 1.460823] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]418gitlab # [ 1.464527] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]419gitlab # [ 1.467366] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]420gitlab # [ 1.469576] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]421gitlab # [ 1.475595] pci 0000:00:01.0: PIIX3: Enabling Passive Release422gitlab # [ 1.480024] pci 0000:00:00.0: Limiting direct PCI/PCI transfers423gitlab # [ 1.488786] ACPI: \_SB_.LNKD: Enabled at IRQ 11424gitlab # [ 1.496876] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 11977 usecs425gitlab # [ 1.502917] PCI: CLS 0 bytes, default 64426gitlab # [ 1.506677] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)427gitlab # [ 1.507137] Trying to unpack rootfs image as initramfs...428gitlab # [ 1.507611] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)429gitlab # [ 1.521676] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229848b50c4, max_idle_ns: 440795322125 ns430gitlab # [ 1.569888] Initialise system trusted keyrings431gitlab # [ 1.573283] workingset: timestamp_bits=40 max_order=21 bucket_order=0432gitlab # [ 1.597425] Key type asymmetric registered433gitlab # [ 1.598742] Asymmetric key parser 'x509' registered434gitlab # [ 1.600305] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)435gitlab # [ 1.603031] io scheduler mq-deadline registered436gitlab # [ 1.604462] io scheduler kyber registered437gitlab # [ 1.609949] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled438gitlab # [ 1.612196] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A439gitlab # [ 1.616212] Linux agpgart interface v0.103440gitlab # [ 1.617691] ACPI: bus type drm_connector registered441gitlab # [ 1.621924] usbcore: registered new interface driver usbserial_generic442gitlab # [ 1.623856] usbserial: USB Serial support registered for generic443gitlab # [ 1.625637] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.444gitlab # [ 1.625637] Please enable it if your BIOS has the CPPC option.445gitlab # [ 1.629557] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled446gitlab # [ 1.631919] drop_monitor: Initializing network drop monitor service447gitlab # [ 1.633949] NET: Registered PF_INET6 protocol family448gitlab # [ 1.636585] Segment Routing with IPv6449gitlab # [ 1.637856] In-situ OAM (IOAM) with IPv6450gitlab # [ 1.641148] IPI shorthand broadcast: enabled451gitlab # [ 1.646389] sched_clock: Marking stable (1387023811, 259332325)->(1730072274, -83716138)452gitlab # [ 1.649390] registered taskstats version 1453gitlab # [ 1.651016] Loading compiled-in X.509 certificates454gitlab # [ 1.661850] Demotion targets for Node 0: null455gitlab # [ 1.663766] Key type .fscrypt registered456gitlab # [ 1.665062] Key type fscrypt-provisioning registered457gitlab # [ 1.666798] ima: No TPM chip found, activating TPM-bypass!458gitlab # [ 1.668501] ima: Allocated hash algorithm: sha1459gitlab # [ 1.669976] ima: No architecture policies found460gitlab # [ 1.671794] PM: Magic number: 2:105:822461gitlab # [ 1.674129] RAS: Correctable Errors collector initialized.462gitlab # [ 1.679579] clk: Disabling unused clocks463gitlab # [ 1.680858] PM: genpd: Disabling unused power domains464gitlab # [ 1.792816] Freeing initrd memory: 27584K465gitlab # [ 1.797477] Freeing unused decrypted memory: 2028K466gitlab # [ 1.801266] Freeing unused kernel image (initmem) memory: 3644K467gitlab # [ 1.803193] Write protecting the kernel read-only data: 32768k468gitlab # [ 1.806423] Freeing unused kernel image (text/rodata gap) memory: 1332K469gitlab # [ 1.809045] Freeing unused kernel image (rodata/data gap) memory: 816K470gitlab # [ 1.861463] x86/mm: Checked W+X mappings: passed, no W+X pages found.471gitlab # [ 1.863510] Run /init as init process472gitlab # [ 1.877713] systemd[1]: Inserted module 'autofs4'473gitlab # [ 1.905083] fuse: init (API version 7.45)474gitlab # [ 1.928453] ACPI: \_SB_.LNKC: Enabled at IRQ 10475gitlab # [ 1.938950] ACPI: \_SB_.LNKA: Enabled at IRQ 10476gitlab # [ 1.944001] ACPI: \_SB_.LNKB: Enabled at IRQ 11477gitlab # [ 1.997433] systemd[1]: Successfully made /usr/ read-only.478gitlab # [ 2.338078] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)479gitlab # [ 2.359670] systemd[1]: Detected virtualization kvm.480gitlab # [ 2.363648] systemd[1]: Detected architecture x86-64.481gitlab # [ 2.367624] systemd[1]: Running in initrd.482gitlab # [ 2.371830] systemd[1]: Initializing machine ID from random generator.483gitlab # [ 2.377059] systemd[1]: Hostname set to <gitlab>.484gitlab # [ 2.455208] systemd[1]: Queued start job for default target Initrd Default Target.485gitlab # [ 2.634505] systemd[1]: Created slice Slice /system/modprobe.486gitlab # [ 2.639557] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.487gitlab # [ 2.645643] systemd[1]: Expecting device /dev/disk/by-label/nix-store...488gitlab # [ 2.650707] systemd[1]: Expecting device /dev/disk/by-label/nixos...489gitlab # [ 2.655677] systemd[1]: Reached target Path Units.490gitlab # [ 2.659702] systemd[1]: Reached target Slice Units.491gitlab # [ 2.663708] systemd[1]: Reached target Swaps.492gitlab # [ 2.667470] systemd[1]: Reached target Timer Units.493gitlab # [ 2.670883] systemd[1]: Listening on D-Bus System Message Bus Socket.494gitlab # [ 2.675165] systemd[1]: Listening on Journal Socket (/dev/log).495gitlab # [ 2.679281] systemd[1]: Listening on Journal Sockets.496gitlab # [ 2.682883] systemd[1]: Listening on udev Control Socket.497gitlab # [ 2.685026] systemd[1]: Listening on udev Kernel Socket.498gitlab # [ 2.686756] systemd[1]: Reached target Socket Units.499gitlab # [ 2.693645] systemd[1]: Starting Create List of Static Device Nodes...500gitlab # [ 2.700193] systemd[1]: Starting Load Kernel Module 9pnet_virtio...501gitlab # [ 2.712904] systemd[1]: Starting Load Kernel Module configfs...502gitlab # [ 2.718147] systemd[1]: Starting Journal Service...503gitlab # [ 2.727954] systemd[1]: Starting Load Kernel Modules...504gitlab # [ 2.733652] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki505gitlab # [ 2.737627] systemd-journald[152]: Collecting audit messages is disabled.506gitlab # [ 2.756948] systemd[1]: Starting Coldplug All udev Devices...507gitlab # [ 2.780928] netfs: FS-Cache loaded508gitlab # [ 2.785103] systemd[1]: Finished Create List of Static Device Nodes.509gitlab # [ 2.791045] 9pnet: Installing 9P2000 support510gitlab # [ 2.791182] systemd[1]: modprobe@configfs.service: Deactivated successfully.511gitlab # [ 2.799481] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.512gitlab # [ 2.799956] systemd[1]: Finished Load Kernel Module configfs.513gitlab # [ 2.806994] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev514gitlab # [ 2.808747] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.515gitlab # [ 2.813776] systemd[1]: Finished Load Kernel Module 9pnet_virtio.516gitlab # [ 2.820272] systemd[1]: Finished Load Kernel Modules.517gitlab # [ 2.822858] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config518gitlab # [ 2.828980] systemd[1]: Starting Apply Kernel Variables...519gitlab # [ 2.571588] systemd-modules-load[155]: Using 4 probe threads520gitlab # [ 2.573738] systemd-modules-l[ 2.832836] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...521gitlab # oad[155]: Inserted module 'virtio_balloon'522gitlab # [ 2.577041] systemd-modules-load[155]: Inserted module 'virtio_gpu'523gitlab # [ 2.837633] systemd[1]: Started Journal Service.524gitlab # [ 2.578826] systemd-modules-load[155]: Inserted module 'dm_mod'525gitlab # [ 2.591398] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.526gitlab # [ 2.595647] systemd[1]: Finished Apply Kernel Variables.527gitlab # [ 2.600470] systemd[1]: Starting Create Static Device Nodes in /dev...528gitlab # [ 2.625064] systemd[1]: Finished Create Static Device Nodes in /dev.529gitlab # [ 2.626641] systemd[1]: Reached target Preparation for Local File Systems.530gitlab # [ 2.628229] systemd[1]: Reached target Local File Systems.531gitlab # [ 2.629754] systemd[1]: Starting Create System Files and Directories...532gitlab # [ 2.632380] systemd[1]: Starting Rule-based Manager for Device Events and Files...533gitlab # [ 2.655495] systemd[1]: Finished Create System Files and Directories.534gitlab # [ 2.661188] systemd-udevd[192]: Using default interface naming scheme 'v260'.535gitlab # [ 2.677570] systemd[1]: Started Rule-based Manager for Device Events and Files.536gitlab # [ 2.693195] systemd[1]: Finished Coldplug All udev Devices.537gitlab # [ 2.694567] systemd[1]: Reached target System Initialization.538gitlab # [ 2.695952] systemd[1]: Reached target Basic System.539gitlab # [ 3.116066] SCSI subsystem initialized540gitlab # [ 3.131250] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12541gitlab # [ 3.134396] serio: i8042 KBD port at 0x60,0x64 irq 1542gitlab # [ 3.135754] serio: i8042 AUX port at 0x60,0x64 irq 12543gitlab # [ 3.142718] uhci_hcd 0000:00:01.2: UHCI Host Controller544gitlab # [ 3.146024] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1545gitlab # [ 3.150462] uhci_hcd 0000:00:01.2: detected 2 ports546gitlab # [ 3.154959] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140547gitlab # [ 3.156978] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18548gitlab # [ 3.159013] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1549gitlab # [ 3.160767] usb usb1: Product: UHCI Host Controller550gitlab # [ 3.161245] virtio_blk virtio4: 4/0/0 default/read/poll queues551gitlab # [ 3.162036] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd552gitlab # [ 3.164938] usb usb1: SerialNumber: 0000:00:01.2553gitlab # [ 3.169893] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)554gitlab # [ 3.175758] hub 1-0:1.0: USB hub found555gitlab # [ 3.176923] hub 1-0:1.0: 2 ports detected556gitlab # [ 3.181974] virtio_blk virtio5: 4/0/0 default/read/poll queues557gitlab # [ 3.190610] virtio_blk virtio5: [vdb] 11450728 512-byte logical blocks (5.86 GB/5.46 GiB)558gitlab # [ 3.210607] scsi host0: ata_piix559gitlab # [ 3.214607] scsi host1: ata_piix560gitlab # [ 3.215663] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0561gitlab # [ 3.217475] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0562gitlab # [ 2.966599] (udev-worker)[215]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.563gitlab # [ 2.971759] (udev-worker)[215]: Network interface NamePolicy= disabled on kernel command line.564gitlab # [ 2.982290] systemd[1]: Starting Virtual Console Setup...565gitlab # [ 2.986470] (udev-worker)[220]: Network interface NamePolicy= disabled on kernel command line.566gitlab # [ 2.989746] systemd[1]: Found device /dev/disk/by-label/nix-store.567gitlab # [ 2.992482] systemd[1]: Found device /dev/disk/by-label/nixos.568gitlab # [ 2.993920] systemd[1]: Reached target Initrd Root Device.569gitlab # [ 2.995458] systemd-vconsole-setup[231]: Configuration of first virtual console was skipped, ignoring remaining ones.570gitlab # [ 2.998227] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...571gitlab # [ 3.014949] systemd[1]: Finished Virtual Console Setup.572gitlab # [ 3.031581] systemd-fsck[240]: nixos: clean, 12/65536 files, 13019/262144 blocks573gitlab # [ 3.038162] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.574gitlab # [ 3.373738] ata2: found unknown device (class 0)575gitlab # [ 3.378576] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100576gitlab # [ 3.385883] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5577gitlab # [ 3.406462] usb 1-1: new full-speed USB device number 2 using uhci_hcd578gitlab # [ 3.428148] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0579gitlab # [ 3.473675] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray580gitlab # [ 3.504608] cdrom: Uniform CD-ROM driver Revision: 3.20581gitlab # [ 3.577633] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00582gitlab # [ 3.582465] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10583gitlab # [ 3.587585] usb 1-1: Product: QEMU USB Tablet584gitlab # [ 3.590667] usb 1-1: Manufacturer: QEMU585gitlab # [ 3.593268] usb 1-1: SerialNumber: 28754-0000:00:01.2-1586gitlab # [ 3.635573] hid: raw HID events driver (C) Jiri Kosina587gitlab # [ 3.649561] usbcore: registered new interface driver usbhid588gitlab # [ 3.652167] usbhid: USB HID core driver589gitlab # [ 3.664265] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2590gitlab # [ 3.668645] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0591gitlab # [ 3.528751] systemd[1]: Mounting /sysroot...592gitlab # [ 3.966505] EXT4-fs (vda): mounted filesystem 80738cfb-64cd-4cd5-bb65-4e8e7f523a6b r/w with ordered data mode. Quota mode: none.593gitlab # [ 3.713299] systemd[1]: Mounted /sysroot.594gitlab # [ 3.715387] systemd[1]: Reached target Initrd Root File System.595gitlab # [ 3.721412] systemd[1]: Mounting /sysroot/nix/.ro-store...596gitlab # [ 3.725537] systemd[1]: Mounting /sysroot/nix/.rw-store...597gitlab # [ 3.735337] systemd[1]: Mounting /sysroot/run...598gitlab # [ 3.740160] systemd[1]: Mounting /sysroot/tmp/shared...599gitlab # [ 4.004015] erofs (device vdb): mounted with root inode @ nid 128.600gitlab # [ 3.745436] systemd[1]: Mounting /sysroot/tmp/xchg...601gitlab # [ 3.747789] systemd[1]: Starting Mountpoints Configured in the Real Root...602gitlab # [ 3.755123] systemd[1]: Mounted /sysroot/nix/.ro-store.603gitlab # [ 3.756473] systemd[1]: Mounted /sysroot/nix/.rw-store.604gitlab # [ 3.757767] systemd[1]: Mounted /sysroot/run.605gitlab # [ 4.019827] 9p: Installing v9fs 9p2000 file system support606gitlab # [ 3.761069] systemd-sysroot-fstab-check[269]: /sysroot should be mounted in the initrd, will request daemon-reload.607gitlab # [ 3.765812] systemd[1]: Starting rw-sysroot-nix-store.service...608gitlab # [ 3.767869] systemd[1]: Reload requested from client PID 269 ('systemd-sysroot') (unit initrd-parse-etc.service)...609gitlab # [ 3.770515] systemd[1]: Reloading...610gitlab # [ 4.030394] systemd[1]: Reloading finished in 263 ms.611gitlab # [ 4.087881] systemd-sysroot-fstab-check[269]: Requesting initrd-fs.target/start/replace...612gitlab # [ 4.233358] systemd[1]: Mounted /sysroot/tmp/shared.613gitlab # [ 4.236810] systemd[1]: Mounted /sysroot/tmp/xchg.614gitlab # [ 4.240563] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.615gitlab # [ 4.246008] systemd[1]: Finished rw-sysroot-nix-store.service.616gitlab # [ 4.260190] systemd-sysroot-fstab-check[269]: Requesting swap.target/start/replace...617gitlab # [ 4.271532] systemd[1]: initrd-parse-etc.service: Deactivated successfully.618gitlab # [ 4.280008] systemd[1]: Finished Mountpoints Configured in the Real Root.619gitlab # [ 4.284426] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.620gitlab # [ 4.298353] systemd[1]: Starting rw-sysroot-nix-store.service...621gitlab # [ 4.324365] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.622gitlab # [ 4.332962] systemd[1]: Finished rw-sysroot-nix-store.service.623gitlab # [ 4.529754] systemd[1]: Mounting /sysroot/nix/store...624gitlab # [ 4.584918] systemd[1]: Mounted /sysroot/nix/store.625gitlab # [ 4.588755] systemd[1]: Reached target Initrd File Systems.626gitlab # [ 4.594642] systemd[1]: Starting Find NixOS closure...627gitlab # [ 4.600210] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...628gitlab # [ 4.635291] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.629gitlab # [ 4.641799] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.630gitlab # [ 4.659250] systemd[1]: Finished Find NixOS closure.631gitlab # [ 4.664550] systemd[1]: Reached target Initrd Default Target.632gitlab # [ 4.672216] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...633gitlab # [ 4.714686] systemd[1]: Stopped target Initrd Default Target.634gitlab # [ 4.718364] systemd[1]: Stopped target Basic System.635gitlab # [ 4.721563] systemd[1]: Stopped target Initrd Root Device.636gitlab # [ 4.724987] systemd[1]: Stopped target Path Units.637gitlab # [ 4.728187] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.638gitlab # [ 4.732632] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.639gitlab # [ 4.737199] systemd[1]: Stopped target Slice Units.640gitlab # [ 4.740379] systemd[1]: Stopped target Socket Units.641gitlab # [ 4.743530] systemd[1]: Stopped target System Initialization.642gitlab # [ 4.746973] systemd[1]: Stopped target Swaps.643gitlab # [ 4.749828] systemd[1]: Stopped target Timer Units.644gitlab # [ 4.752686] systemd[1]: dbus.socket: Deactivated successfully.645gitlab # [ 4.755643] systemd[1]: Closed D-Bus System Message Bus Socket.646gitlab # [ 4.758448] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.647gitlab # [ 4.761875] systemd[1]: Stopped Find NixOS closure.648gitlab # [ 4.764254] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio649gitlab # [ 4.769101] systemd[1]: Starting rw-sysroot-nix-store.service...650gitlab # [ 4.771513] systemd[1]: systemd-sysctl.service: Deactivated successfully.651gitlab # [ 4.773194] systemd[1]: Stopped Apply Kernel Variables.652gitlab # [ 4.774488] systemd[1]: systemd-modules-load.service: Deactivated successfully.653gitlab # [ 4.776256] systemd[1]: Stopped Load Kernel Modules.654gitlab # [ 4.778822] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.655gitlab # [ 4.780739] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.656gitlab # [ 4.786525] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.657gitlab # [ 4.788276] systemd[1]: Stopped Create System Files and Directories.658gitlab # [ 4.791618] systemd[1]: Stopped target Local File Systems.659gitlab # [ 4.793005] systemd[1]: Stopped target Preparation for Local File Systems.660gitlab # [ 4.794635] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.661gitlab # [ 4.796328] systemd[1]: Stopped Coldplug All udev Devices.662gitlab # [ 4.798843] systemd[1]: Stopping Rule-based Manager for Device Events and Files...663gitlab # [ 4.800626] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.664gitlab # [ 4.802794] systemd[1]: Stopped Virtual Console Setup.665gitlab # [ 4.811437] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.666gitlab # [ 4.816484] systemd[1]: Finished rw-sysroot-nix-store.service.667gitlab # [ 4.822246] systemd[1]: systemd-udevd.service: Deactivated successfully.668gitlab # [ 4.827277] systemd[1]: Stopped Rule-based Manager for Device Events and Files.669gitlab # [ 4.829065] systemd[1]: systemd-udevd.service: Consumed 1.108s CPU time over 2.190s wall clock time, 26.2M memory peak.670gitlab # [ 4.838414] systemd[1]: initrd-cleanup.service: Deactivated successfully.671gitlab # [ 4.841725] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.672gitlab # [ 4.849616] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.673gitlab # [ 4.851349] systemd[1]: Closed udev Control Socket.674gitlab # [ 4.854367] systemd[1]: Starting Cleanup udev Database...675gitlab # [ 4.855752] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.676gitlab # [ 4.857566] systemd[1]: Stopped Create Static Device Nodes in /dev.677gitlab # [ 4.859089] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.678gitlab # [ 4.861055] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.679gitlab # [ 4.862812] systemd[1]: kmod-static-nodes.service: Deactivated successfully.680gitlab # [ 4.864511] systemd[1]: Stopped Create List of Static Device Nodes.681gitlab # [ 4.901459] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.682gitlab # [ 4.907462] systemd[1]: Finished Cleanup udev Database.683gitlab # [ 4.916509] systemd[1]: Reached target Switch Root.684gitlab # [ 4.922498] systemd[1]: Starting NixOS Activation...685gitlab # [ 5.067613] initrd-nixos-activation-start[570]: booting system configuration /nix/store/15f597533s2qv6mxxjladp3li5g5zvjg-nixos-system-gitlab-test686gitlab # [ 5.115917] initrd-nixos-activation-start[570]: running activation script...687gitlab # [ 5.332978] initrd-nixos-activation-start[593]: setting up /etc...688gitlab # [ 5.475273] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.689gitlab # [ 5.482507] systemd[1]: Finished NixOS Activation.690gitlab # [ 5.498684] systemd[1]: Starting Switch Root...691gitlab # [ 5.540496] systemd[1]: Switching root.692gitlab # [ 6.235825] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).693gitlab # [ 6.358724] NET: Registered PF_VSOCK protocol family694gitlab # [ 6.728172] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)695gitlab # [ 6.744175] systemd[1]: Detected virtualization kvm.696gitlab # [ 6.747288] systemd[1]: Detected architecture x86-64.697gitlab # [ 6.750649] systemd[1]: Detected first boot.698gitlab # [ 6.754798] systemd[1]: Initializing machine ID from random generator.699gitlab # [ 6.894475] systemd[1]: bpf-restrict-fs: LSM BPF program attached700gitlab # [ 6.958877] systemd[1]: Applying preset policy.701gitlab # [ 7.136086] systemd[1]: Populated /etc with preset unit settings.702gitlab # [ 7.262984] systemd[1]: initrd-switch-root.service: Deactivated successfully.703gitlab # [ 7.265346] systemd[1]: Stopped initrd-switch-root.service.704gitlab # [ 7.268888] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.705gitlab # [ 7.272289] systemd[1]: Created slice Slice /system/getty.706gitlab # [ 7.274503] systemd[1]: Created slice GitLab DevOps Platform Slice.707gitlab # [ 7.276813] systemd[1]: Created slice User and Session Slice.708gitlab # [ 7.278410] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.709gitlab # [ 7.280501] systemd[1]: Started Forward Password Requests to Wall Directory Watch.710gitlab # [ 7.282374] systemd[1]: Expecting device /dev/hvc0...711gitlab # [ 7.283698] systemd[1]: Expecting device /dev/ttyS0...712gitlab # [ 7.285037] systemd[1]: Reached target Local Encrypted Volumes.713gitlab # [ 7.286556] systemd[1]: Stopped target initrd-fs.target.714gitlab # [ 7.287933] systemd[1]: Stopped target initrd-root-fs.target.715gitlab # [ 7.289387] systemd[1]: Stopped target initrd-switch-root.target.716gitlab # [ 7.290941] systemd[1]: Reached target Virtual Machines and Containers.717gitlab # [ 7.292617] systemd[1]: Reached target Path Units.718gitlab # [ 7.293909] systemd[1]: Reached target Remote File Systems.719gitlab # [ 7.295370] systemd[1]: Reached target Slice Units.720gitlab # [ 7.296663] systemd[1]: Reached target Swaps.721gitlab # [ 7.298924] systemd[1]: Listening on Process Core Dump Socket.722gitlab # [ 7.301008] systemd[1]: Listening on Credential Encryption/Decryption.723gitlab # [ 7.304497] systemd[1]: Starting Journal Log Access Socket...724gitlab # [ 7.306686] systemd[1]: Listening on Journal Audit Socket.725gitlab # [ 7.308372] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.726gitlab # [ 7.310238] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki727gitlab # [ 7.312699] systemd[1]: Listening on udev Control Socket.728gitlab # [ 7.317790] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 7.320719] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 7.323715] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 7.327429] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 7.336240] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 7.338941] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 7.344121] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 7.348297] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 7.354599] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 7.354794] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 7.361428] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 7.365540] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 7.386441] systemd[1]: Starting Journal Service...741gitlab # [ 7.390522] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 7.395373] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 7.401506] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 7.403215] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki745gitlab # [ 7.411467] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 7.417279] systemd[1]: Listening on Journal Log Access Socket.747gitlab # [ 7.419086] systemd[1]: Mounted Huge Pages File System.748gitlab # [ 7.419704] systemd-journald[805]: Collecting audit messages is enabled.749gitlab # [ 7.421174] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 7.424600] systemd[1]: Mounted Kernel Debug File System.751gitlab # [ 7.424825] systemd[1]: Mounted Kernel Trace File System.752gitlab # [ 7.428499] systemd[1]: Finished Create List of Static Device Nodes.753gitlab # [ 7.431240] systemd[1]: modprobe@configfs.service: Deactivated successfully.754gitlab # [ 7.434446] EXT4-fs (vda): re-mounted 80738cfb-64cd-4cd5-bb65-4e8e7f523a6b.755gitlab # [ 7.439896] systemd[1]: Finished Load Kernel Module configfs.756gitlab # [ 7.183440] systemd[1]: Queue[ 7.442639] loop: module loaded757gitlab # d start job for default target Multi-User System.758gitlab # [ 7.186485] systemd[1]: Unnecessary job was r[ 7.445885] systemd[1]: Started Journal Service.759gitlab # emoved for /dev/vdb.760gitlab # [ 7.189436] systemd[1]: systemd-journald.service: Deactivated successfully.761gitlab # [ 7.191863] systemd-modules-load[806]: Using 3 probe threads762gitlab # [ 7.193601] systemd-modules-load[806]: Inserted module 'tls'763gitlab # [ 7.195415] systemd-modules-load[806]: Inserted module 'loop'764gitlab # [ 7.197141] systemd[1]: Mounted FUSE Control File System.765gitlab # [ 7.199349] systemd[1]: Finished Remount Root and Kernel File Systems.766gitlab # [ 7.201296] systemd[1]: Finished Load Kernel Modules.767gitlab # [ 7.204568] systemd[1]: Mounting Kernel Configuration File System...768gitlab # [ 7.206425] systemd[1]: Starting Firewall...769gitlab # [ 7.209428] systemd[1]: Starting Flush Journal to Persistent Storage...770gitlab # [ 7.212205] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore771gitlab # [ 7.219258] systemd[1]: Starting Load/Save OS Random Seed...772gitlab # [ 7.223081] systemd[1]: Starting Apply Kernel Variables...773gitlab # [ 7.224988] systemd-oomd[807]: No swap; memory pressure usage will be degraded774gitlab # [ 7.227969] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...775gitlab # [ 7.232101] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki776gitlab # [ 7.494384] systemd-journald[805]: Received client request to flush runtime journal.777gitlab # [ 7.308567] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.778gitlab # [ 7.311282] systemd[1]: Mounted Kernel Configuration File System.779gitlab # [ 7.313080] systemd[1]: Finished Load/Save OS Random Seed.780gitlab # [ 7.315870] systemd[1]: Reached target First Boot Complete.781gitlab # [ 7.317532] systemd[1]: Finished Apply Kernel Variables.782gitlab # [ 7.319082] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.783gitlab # [ 7.321209] systemd[1]: Starting Create Static Device Nodes in /dev...784gitlab # [ 7.322879] systemd[1]: Finished Create Static Device Nodes in /dev.785gitlab # [ 7.324406] systemd[1]: Reached target Preparation for Local File Systems.786gitlab # [ 7.326097] systemd[1]: Starting Rule-based Manager for Device Events and Files...787gitlab # [ 7.328057] systemd[1]: Finished Flush Journal to Persistent Storage.788gitlab # [ 7.339308] systemd-udevd[854]: Using default interface naming scheme 'v260'.789gitlab # [ 7.378279] systemd[1]: Started Rule-based Manager for Device Events and Files.790gitlab # [ 7.385371] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 7.500123] systemd[1]: Finished Firewall.792gitlab # [ 7.502693] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 7.532797] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 7.544919] (udev-worker)[941]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.795gitlab # [ 7.548832] (udev-worker)[941]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 7.551265] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.797gitlab # [ 7.554057] (udev-worker)[944]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 7.565777] systemd[1]: Condition check resulted in Virtio network device being skipped.799gitlab # [ 7.568913] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore800gitlab # [ 7.572161] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67801gitlab # [ 7.577323] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore802gitlab # [ 7.580229] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki803gitlab # [ 7.583582] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki804gitlab # [ 7.878879] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0805gitlab # [ 7.880829] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD806gitlab # [ 7.884583] mousedev: PS/2 mouse device common for all mice807gitlab # [ 7.884766] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3808gitlab # [ 7.903548] ACPI: button: Power Button [PWRF]809gitlab # [ 7.940675] Floppy drive(s): fd0 is 2.88M AMI BIOS810gitlab # [ 7.944100] rtc_cmos 00:05: RTC can wake from S4811gitlab # [ 7.945118] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console812gitlab # [ 7.951412] parport_pc 00:03: reported by Plug and Play ACPI813gitlab # [ 7.951595] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]814gitlab # [ 7.954103] FDC 0 is a S82078B815gitlab # [ 7.955530] rtc_cmos 00:05: registered as rtc0816gitlab # [ 7.955702] rtc_cmos 00:05: setting system clock to 2026-07-09T06:48:56 UTC (1783579736)817gitlab # [ 7.955881] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs818gitlab # [ 7.959124] systemd-journald[805]: Time jumped backwards, rotating.819gitlab # [ 7.962126] Console: switching to colour dummy device 80x25820gitlab # [ 7.973534] [drm] Found bochs VGA, ID 0xb0c5.821gitlab # [ 7.976376] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.822gitlab # [ 7.982430] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4823gitlab # [ 7.984208] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6824gitlab # [ 7.984241] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic825gitlab # [ 7.987257] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0826gitlab # [ 7.996473] fbcon: bochs-drmdrmfb (fb0) is primary device827gitlab # [ 8.002934] Console: switching to colour frame buffer device 160x50828gitlab # [ 8.006113] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device829gitlab # [ 8.010799] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5830gitlab # [ 7.811121] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 7.827868] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.832gitlab # [ 7.832426] systemd[1]: Stopped Virtual Console Setup.833gitlab # [ 7.859339] systemd[1]: Starting Virtual Console Setup...834gitlab # [ 7.873092] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.835gitlab # [ 7.875126] systemd[1]: Stopped Virtual Console Setup.836gitlab # [ 7.881908] systemd[1]: Starting Virtual Console Setup...837gitlab # [ 8.158662] ppdev: user-space parallel port driver838gitlab # [ 8.185742] kvm_amd: TSC scaling supported839gitlab # [ 8.186883] kvm_amd: Nested Virtualization enabled840gitlab # [ 8.188436] kvm_amd: Nested Paging enabled841gitlab # [ 8.189587] kvm_amd: LBR virtualization supported842gitlab # [ 8.190767] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 8.192083] kvm_amd: Virtual GIF supported844gitlab # [ 8.193081] kvm_amd: Virtual NMI enabled845gitlab # [ 8.242843] EDAC MC: Ver: 3.0.0846gitlab # [ 7.999724] systemd-vconsole-setup[978]: Configuration of first virtual console was skipped, ignoring remaining ones.847gitlab # [ 8.005043] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 8.007745] systemd[1]: Mounting /run/wrappers...849gitlab # [ 8.029799] systemd[1]: Mounted /run/wrappers.850gitlab # [ 8.031148] systemd[1]: Reached target Local File Systems.851gitlab # [ 8.032607] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 8.034378] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 8.035988] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 8.037628] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 8.039978] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 8.044869] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 8.065889] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 8.069385] systemd[1]: Finished Save Transient machine-id to Disk.859gitlab # [ 8.077703] systemd[1]: Finished Create System Files and Directories.860gitlab # [ 8.080101] systemd[1]: Starting Rebuild Journal Catalog...861gitlab # [ 8.082429] systemd[1]: Starting Record System Boot/Shutdown in UTMP...862gitlab # [ 8.104951] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 8.117488] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 8.121893] systemd[1]: Starting Update is Completed...865gitlab # [ 8.139544] systemd[1]: Finished Update is Completed.866gitlab # [ 8.182403] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 8.184285] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 8.185649] systemd[1]: suid-sgid-wrappers.service: Consumed 120ms CPU time over 147ms wall clock time, 5M memory peak, 1.1M read from disk.869gitlab # [ 8.446802] systemd[1]: Finished register-nix-paths.service.870gitlab # [ 8.449457] systemd[1]: Reached target System Initialization.871gitlab # [ 8.451700] systemd[1]: Started Discard unused filesystem blocks once a week.872gitlab # [ 8.454336] systemd[1]: Started logrotate.timer.873gitlab # [ 8.456474] systemd[1]: Started Daily Cleanup of Temporary Directories.874gitlab # [ 8.458788] systemd[1]: Reached target Timer Units.875gitlab # [ 8.460791] systemd[1]: Listening on D-Bus System Message Bus Socket.876gitlab # [ 8.463363] systemd[1]: Listening on Nix Daemon Socket.877gitlab # [ 8.468768] systemd[1]: Starting nixbot CI service socket...878gitlab # [ 8.471141] systemd[1]: Listening on Hostname Service Socket.879gitlab # [ 8.473503] systemd[1]: Listening on nixbot CI service socket.880gitlab # [ 8.475885] systemd[1]: Reached target Socket Units.881gitlab # [ 8.478442] systemd[1]: Starting D-Bus System Message Bus...882gitlab # [ 8.546269] dbus-broker-launch[1070]: Looking up NSS user entry for 'systemd-timesync'...883gitlab # [ 8.551691] dbus-broker-launch[1070]: NSS returned no entry for 'systemd-timesync'884gitlab # [ 8.556605] dbus-broker-launch[1070]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"885gitlab # [ 8.565574] systemd[1]: Started D-Bus System Message Bus.886gitlab # [ 8.578703] dbus-broker-launch[1070]: Ready887gitlab # [ 8.584890] systemd[1]: Reached target Basic System.888gitlab # [ 8.588177] systemd[1]: Started backdoor.service.889gitlab # [ 8.591291] systemd[1]: Starting gitlab-config.service...890gitlab # [ 8.594386] systemd[1]: Starting Import lastlog data into lastlog2 database...891gitlab # [ 8.599693] systemd[1]: Starting Logrotate configuration check...892gitlab # [ 8.606368] systemd[1]: Starting Name Service Cache Daemon (nsncd)...893gitlab # [ 8.611963] systemd[1]: Starting Post-Boot Actions...894gitlab # [ 8.618510] systemd[1]: Started Reset console on configuration changes.895gitlab # [ 8.622064] logrotate[1075]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.896gitlab # [ 8.630360] logrotate[1075]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf897gitlab # connecting to host...898gitlab # [ 8.634036] logrotate[1075]: note: 'monthly' overrides previously specified 'weekly'899gitlab # [ 8.636804] logrotate[1075]: Reading state from file: /var/lib/logrotate.status900gitlab # [ 8.638790] logrotate[1075]: state file /var/lib/logrotate.status does not exist901gitlab # [ 8.640692] logrotate[1075]: Allocating hash table for state file, size 64 entries902gitlab # [ 8.642738] logrotate[1075]: Handling 4 logs903gitlab # [ 8.643894] logrotate[1075]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed904gitlab: Guest shell says: b'Spawning backdoor root shell...\n'905gitlab: connected to guest root shell906gitlab: (connecting took 9.50 seconds)907gitlab: (finished: waiting for the VM to finish booting, in 9.50 seconds)908gitlab # [ 8.647504] logrotate[1075]: considering log /var/log/btmp909gitlab # [ 8.648933] logrotate[1075]: Creating new state910gitlab # [ 8.650973] logrotate[1075]: Now: 2026-07-09 06:48911gitlab # [ 8.652224] logrotate[1075]: Last rotated at 2026-07-09 06:00912gitlab # [ 8.655603] logrotate[1075]: log does not need rotating (log has already been rotated)913gitlab # [ 8.656853] logrotate[1075]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed914gitlab # [ 8.658649] logrotate[1075]: considering log /var/log/wtmp915gitlab # [ 8.659770] logrotate[1075]: Creating new state916gitlab # [ 8.661228] logrotate[1075]: Now: 2026-07-09 06:48917gitlab # [ 8.662819] logrotate[1075]: Last rotated at 2026-07-09 06:00918gitlab # [ 8.663982] logrotate[1075]: log does not need rotating (log has already been rotated)919gitlab # [ 8.666933] logrotate[1075]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed920gitlab # [ 8.668812] logrotate[1075]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1075)921gitlab # [ 8.669609] logrotate[1075]: considering log /var/gitlab/state/log/*.log922gitlab # [ 8.670474] logrotate[1075]: log /var/gitlab/state/log/*.log does not exist -- skipping923gitlab # [ 8.671635] logrotate[1075]: Creating new state924gitlab # [ 8.673136] logrotate[1075]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1075)925gitlab # [ 8.674042] logrotate[1075]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed926gitlab # [ 8.674586] logrotate[1075]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1075)927gitlab # [ 8.675707] logrotate[1075]: considering log /var/log/nginx/*.log928gitlab # [ 8.677205] logrotate[1075]: log /var/log/nginx/*.log does not exist -- skipping929gitlab # [ 8.677992] logrotate[1075]: Creating new state930gitlab # [ 8.679192] logrotate[1075]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1075)931gitlab # [ 8.680187] systemd[1]: Starting resolvconf update...932gitlab # [ 8.681426] nsncd[1076]: Jul 09 06:48:57.466 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"933gitlab # [ 8.683131] systemd[1]: Finished Logrotate configuration check.934gitlab # [ 8.684952] systemd[1]: Finished Post-Boot Actions.935gitlab # [ 8.687593] systemd[1]: Finished Import lastlog data into lastlog2 database.936gitlab # [ 8.688816] systemd[1]: Started Name Service Cache Daemon (nsncd).937gitlab # [ 8.690330] systemd[1]: Reached target Host and Network Name Lookups.938gitlab # [ 8.692325] systemd[1]: Reached target User and Group Name Lookups.939gitlab # [ 8.693112] systemd[1]: Starting User Login Management...940gitlab # [ 8.707935] systemd[1]: Stopped target Host and Network Name Lookups.941gitlab # [ 8.709208] systemd[1]: Stopping Host and Network Name Lookups...942gitlab # [ 8.710473] systemd[1]: Stopped target User and Group Name Lookups.943gitlab # [ 8.711400] systemd[1]: Stopping User and Group Name Lookups...944gitlab # [ 8.713639] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...945gitlab # [ 8.715794] systemd[1]: nscd.service: Deactivated successfully.946gitlab # [ 8.718142] systemd[1]: Stopped Name Service Cache Daemon (nsncd).947gitlab # [ 8.719811] systemd[1]: nscd.service: Consumed 21ms CPU time over 110ms wall clock time, 7.8M memory peak, 5.6M read from disk.948gitlab # [ 8.732306] systemd[1]: Starting Name Service Cache Daemon (nsncd)...949gitlab # [ 8.752737] systemd[1]: Started Name Service Cache Daemon (nsncd).950gitlab # [ 8.754175] nsncd[1156]: Jul 09 06:48:57.556 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"951gitlab # [ 8.756934] systemd[1]: Reached target Host and Network Name Lookups.952gitlab # [ 8.758187] systemd[1]: Reached target User and Group Name Lookups.953gitlab # [ 8.761958] systemd-logind[1118]: New seat seat0.954gitlab # [ 8.767171] systemd-logind[1118]: Watching system buttons on /dev/input/event2 (Power Button)955gitlab # [ 8.768386] systemd-logind[1118]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)956gitlab # [ 8.769404] systemd-logind[1118]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)957gitlab # [ 8.776329] systemd[1]: Started User Login Management.958gitlab # [ 8.777565] systemd[1]: Starting linger-users.service...959gitlab # [ 8.778908] systemd[1]: Finished resolvconf update.960gitlab # [ 8.779685] systemd[1]: Reached target Preparation for Network.961gitlab # [ 8.785695] systemd[1]: Starting DHCP Client...962gitlab # [ 8.789435] systemd[1]: Starting Address configuration of eth1...963gitlab # [ 8.791315] systemd[1]: Starting Extra networking commands....964gitlab # [ 8.794274] systemd[1]: linger-users.service: Deactivated successfully.965gitlab # [ 8.800946] systemd[1]: Finished linger-users.service.966gitlab # [ 8.824607] network-addresses-eth1-start[1187]: adding address 192.168.1.1/24... done967gitlab # [ 8.830161] network-addresses-eth1-start[1187]: adding address 2001:db8:1::1/64... done968gitlab # [ 8.836640] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 8.869761] systemd[1]: Finished Extra networking commands..970gitlab # [ 8.871601] systemd[1]: Reached target Network.971gitlab # [ 8.876809] systemd[1]: Starting gitlab-workhorse.service...972gitlab # [ 8.879639] dhcpcd[1202]: dhcpcd-10.3.2 starting973gitlab # [ 8.882344] systemd[1]: Starting Nginx Web Server...974gitlab # [ 8.885766] dhcpcd[1248]: dev: loaded udev975gitlab # [ 8.887053] systemd[1]: Starting PostgreSQL Server...976gitlab # [ 8.891102] systemd[1]: Starting Redis Server - redis-gitlab...977gitlab # [ 9.152811] 8021q: 802.1Q VLAN Support v1.8978gitlab # [ 9.153773] 8021q: adding VLAN 0 to HW filter on device eth1979gitlab # [ 8.898326] systemd[1]: Starting Permit User Sessions...980gitlab # [ 8.913292] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 8.918486] systemd[1]: Finished Permit User Sessions.982gitlab # [ 8.924874] systemd[1]: Started Getty on tty1.983gitlab # [ 8.927866] systemd[1]: Reached target Login Prompts.984gitlab # [ 9.254703] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 9.265225] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 9.267383] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 9.269623] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 9.271472] cfg80211: failed to load regulatory.db989gitlab # [ 9.027233] nginx-pre-start[1268]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok990gitlab # [ 9.029690] nginx-pre-start[1268]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful991gitlab # [ 9.037463] systemd[1]: Started Nginx Web Server.992gitlab # [ 9.050241] redis-server[1270]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.993gitlab # [ 9.051956] redis-server[1270]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo994gitlab # [ 9.057335] redis-server[1270]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1270, just started995gitlab # [ 9.058775] systemd[1]: Started Redis Server - redis-gitlab.996gitlab # [ 9.059940] redis-server[1270]: Configuration loaded997gitlab # [ 9.061082] redis[1270]: monotonic clock: POSIX clock_gettime998gitlab # [ 9.061919] redis[1270]: Running mode=standalone, port=0.999gitlab # [ 9.063172] redis[1270]: Server initialized1000gitlab # [ 9.064193] redis[1270]: Ready to accept connections unix1001gitlab # [ 9.065070] redis[1270]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1002gitlab # [ 9.330716] 8021q: adding VLAN 0 to HW filter on device eth01003gitlab # [ 9.073759] dhcpcd[1248]: eth0: waiting for carrier1004gitlab # [ 9.074891] dhcpcd[1248]: eth0: carrier acquired1005gitlab # [ 9.080181] dhcpcd[1248]: DUID 00:01:00:01:31:e2:00:d9:52:54:00:12:34:561006gitlab # [ 9.081490] dhcpcd[1248]: eth0: IAID 00:12:34:561007gitlab # [ 9.083400] dhcpcd[1248]: eth0: adding address fe80::5054:ff:fe12:34561008gitlab # [ 9.111953] postgresql-pre-start[1277]: The files belonging to this database system will be owned by user "postgres".1009gitlab # [ 9.113370] postgresql-pre-start[1277]: This user must also own the server process.1010gitlab # [ 9.116195] postgresql-pre-start[1277]: The database cluster will be initialized with locale "en_US.UTF-8".1011gitlab # [ 9.116957] postgresql-pre-start[1277]: The default database encoding has accordingly been set to "UTF8".1012gitlab # [ 9.118130] postgresql-pre-start[1277]: The default text search configuration will be set to "english".1013gitlab # [ 9.119047] postgresql-pre-start[1277]: Data page checksums are disabled.1014gitlab # [ 9.120131] postgresql-pre-start[1277]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1015gitlab # [ 9.121395] postgresql-pre-start[1277]: creating subdirectories ... ok1016gitlab # [ 9.122819] postgresql-pre-start[1277]: selecting dynamic shared memory implementation ... posix1017gitlab # [ 9.146608] postgresql-pre-start[1277]: selecting default "max_connections" ... 1001018gitlab # [ 9.192846] postgresql-pre-start[1277]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 9.383305] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 9.584381] postgresql-pre-start[1277]: selecting default time zone ... UTC1021gitlab # [ 9.586711] postgresql-pre-start[1277]: creating configuration files ... ok1022gitlab # [ 9.668715] gitlab-workhorse[1306]: time="2026-07-09T06:48:58Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41023gitlab # [ 9.674412] gitlab-workhorse[1306]: time="2026-07-09T06:48:58Z" level=info msg="Using redis/go-redis"1024gitlab # [ 9.679110] gitlab-workhorse[1306]: time="2026-07-09T06:48:58Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 9.772993] postgresql-pre-start[1277]: running bootstrap script ... ok1026gitlab # [ 9.799337] dhcpcd[1248]: eth0: soliciting a DHCP lease1027gitlab # [ 10.066517] NET: Registered PF_PACKET protocol family1028gitlab # [ 9.814869] dhcpcd[1248]: eth0: offered 10.0.2.15 from 10.0.2.21029gitlab # [ 9.820266] dhcpcd[1248]: eth0: probing address 10.0.2.15/241030gitlab # [ 10.257525] postgresql-pre-start[1277]: performing post-bootstrap initialization ... ok1031gitlab # [ 10.635537] postgresql-pre-start[1277]: syncing data to disk ... ok1032gitlab # [ 10.639227] postgresql-pre-start[1277]: initdb: warning: enabling "trust" authentication for local connections1033gitlab # [ 10.641869] postgresql-pre-start[1277]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1034gitlab # [ 10.646269] postgresql-pre-start[1277]: Success. You can now start the database server using:1035gitlab # [ 10.649265] postgresql-pre-start[1277]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1036gitlab # [ 10.758706] postgres[1332]: [1332] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1037gitlab # [ 10.765934] postgres[1332]: [1332] LOG: listening on IPv6 address "::1", port 54321038gitlab # [ 10.769264] postgres[1332]: [1332] LOG: listening on IPv4 address "127.0.0.1", port 54321039gitlab # [ 10.778304] postgres[1332]: [1332] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1040gitlab # [ 10.795362] postgres[1338]: [1338] LOG: database system was shut down at 2026-07-09 06:48:59 GMT1041gitlab # [ 10.805958] postgres[1332]: [1332] LOG: database system is ready to accept connections1042gitlab # [ 10.812801] systemd[1]: Started PostgreSQL Server.1043gitlab # [ 10.820631] systemd[1]: Starting PostgreSQL Setup Scripts...1044gitlab # [ 10.998557] postgresql-setup-start[1358]: CREATE DATABASE1045gitlab # [ 11.039612] postgresql-setup-start[1368]: CREATE ROLE1046gitlab # [ 11.055492] postgresql-setup-start[1370]: ALTER DATABASE1047gitlab # [ 11.091199] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1363]: mkdir -p /var/gitlab/state/home/.ssh: OK1048gitlab # [ 11.096988] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1363]: chmod 700 /var/gitlab/state/home/.ssh: OK1049gitlab # [ 11.103331] postgresql-setup-start[1375]: CREATE ROLE1050gitlab # [ 11.106694] systemd[1]: Finished PostgreSQL Setup Scripts.1051gitlab # [ 11.110219] systemd[1]: Reached target PostgreSQL.1052gitlab # [ 11.112745] systemd[1]: Starting gitlab-postgresql.service...1053gitlab # [ 11.142581] systemd[1]: Finished gitlab-config.service.1054gitlab # [ 11.147507] systemd[1]: Started gitaly.service.1055gitlab # [ 11.149514] systemd[1]: Starting gitlab-db-config.service...1056gitlab # [ 11.213892] gitlab-postgresql-start[1394]: CREATE DATABASE1057gitlab # [ 11.260114] gitlab-postgresql-start[1404]: CREATE EXTENSION1058gitlab # [ 11.300507] gitlab-postgresql-start[1406]: CREATE EXTENSION1059gitlab # [ 11.304522] systemd[1]: Finished gitlab-postgresql.service.1060gitlab # [ 11.420504] dhcpcd[1248]: eth0: soliciting an IPv6 router1061gitlab # [ 11.423589] dhcpcd[1248]: eth0: Router Advertisement from fe80::21062gitlab # [ 11.425139] dhcpcd[1248]: eth0: adding address fec0::5054:ff:fe12:3456/641063gitlab # [ 11.427105] dhcpcd[1248]: eth0: adding route to fec0::/641064gitlab # [ 11.428759] dhcpcd[1248]: eth0: adding default route via fe80::21065gitlab # [ 11.695457] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13971066gitlab # [ 11.697627] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13971067gitlab # [ 11.699537] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13971068gitlab # [ 11.702420] gitaly[1397]: time="2026-07-09T06:49:00.494Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13971069gitlab # [ 11.703950] gitaly[1397]: time="2026-07-09T06:49:00.495Z" level=info msg="Starting Gitaly" pid=1397 version=18.11.41070gitlab # [ 11.899498] gitaly[1397]: time="2026-07-09T06:49:00.702Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=207 pid=13971071gitlab # [ 11.901126] gitaly[1397]: time="2026-07-09T06:49:00.703Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13971072gitlab # [ 11.901662] gitaly[1397]: time="2026-07-09T06:49:00.703Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13971073gitlab # [ 11.902807] gitaly[1397]: time="2026-07-09T06:49:00.703Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1397/git-exec-1981981585.d/git pid=13971074gitlab # [ 11.906141] gitaly[1397]: time="2026-07-09T06:49:00.709Z" level=info msg="finished detecting git version" duration_ms=5 pid=13971075gitlab # [ 11.906990] gitaly[1397]: time="2026-07-09T06:49:00.709Z" level=info msg="using Git version" pid=1397 version=2.53.g1076gitlab # [ 11.907915] gitaly[1397]: time="2026-07-09T06:49:00.709Z" level=info msg="clearing disk cache object folder" pid=1397 storage=default1077gitlab # [ 11.908984] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4229726837 pid=1397 storage=default1078gitlab # [ 11.910196] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4229726837 pid=1397 storage=default1079gitlab # [ 11.910803] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13971080gitlab # [ 11.911859] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13971081gitlab # [ 11.913049] gitaly[1397]: time="2026-07-09T06:49:00.710Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4229726837 clear_duration_ms=0 pid=1397 storage=default1082gitlab # [ 11.977189] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="counting repositories" pid=13971083gitlab # [ 11.978839] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="starting to count repositories" pid=1397 storage_path=/var/gitlab/state/repositories1084gitlab # [ 11.980324] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1397 storage_path=/var/gitlab/state/repositories1085gitlab # [ 11.981966] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13971086gitlab # [ 11.983331] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13971087gitlab # [ 11.983991] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="finished tempdir cleaner walk" pid=1397 storage=default time_ms=01088gitlab # [ 11.985046] gitaly[1397]: time="2026-07-09T06:49:00.780Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1397 schema=unix1089gitlab # [ 11.986087] gitaly[1397]: time="2026-07-09T06:49:00.781Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1397/sock.d/intern pid=1397 schema=unix1090gitlab # [ 12.005133] gitaly[1397]: time="2026-07-09T06:49:00.808Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13971091gitlab # [ 12.005988] gitaly[1397]: time="2026-07-09T06:49:00.808Z" level=info msg="Started Gitaly" duration_ms=313 pid=13971092gitlab # [ 12.011225] gitaly[1397]: time="2026-07-09T06:49:00.814Z" level=info msg="maintenance: daily scheduled" pid=1397 scheduled="2026-07-09 12:00:00 +0000 UTC"1093gitlab # [ 12.626644] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1094gitlab # [ 12.631695] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1095gitlab # [ 12.652763] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1096gitlab # [ 13.880240] gitaly[1397]: time="2026-07-09T06:49:02.682Z" level=info msg="License database preloaded" duration_ms=2186 pid=13971097gitlab # [ 15.173582] dhcpcd[1248]: eth0: leased 10.0.2.15 for 86400 seconds1098gitlab # [ 15.177730] dhcpcd[1248]: eth0: adding route to 10.0.2.0/241099gitlab # [ 15.181161] dhcpcd[1248]: eth0: adding default route via 10.0.2.21100gitlab # [ 15.312960] systemd[1]: Started DHCP Client.1101gitlab # [ 15.318834] systemd[1]: Reached target Network is Online.1102gitlab # [ 44.007829] postgres[1770]: [1770] ERROR: relation "feature_gates" does not exist at character 5231103gitlab # [ 44.012554] postgres[1770]: [1770] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1104gitlab # [ 44.015884] postgres[1770]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1105gitlab # [ 44.021814] postgres[1770]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1106gitlab # [ 44.026750] postgres[1770]: attidentity AS identity,1107gitlab # [ 44.030314] postgres[1770]: attgenerated as attgenerated1108gitlab # [ 44.034361] postgres[1770]: FROM pg_attribute a1109gitlab # [ 44.037565] postgres[1770]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1110gitlab # [ 44.042159] postgres[1770]: LEFT JOIN pg_type t ON a.atttypid = t.oid1111gitlab # [ 44.046283] postgres[1770]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1112gitlab # [ 44.049718] postgres[1770]: WHERE a.attrelid = '"feature_gates"'::regclass1113gitlab # [ 44.053514] postgres[1770]: AND a.attnum > 0 AND NOT a.attisdropped1114gitlab # [ 44.057101] postgres[1770]: ORDER BY a.attnum1115gitlab # [ 44.059475] postgres[1770]:1116gitlab # [ 44.242113] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1400]: W, [2026-07-09T06:49:33.044452 #1400] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1117gitlab # [ 49.732770] gitlab-workhorse[1306]: time="2026-07-09T06:49:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1118gitlab # [ 49.891736] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1830]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1119gitlab # [ 49.895331] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1830]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1120gitlab # [ 59.743426] gitlab-workhorse[1306]: time="2026-07-09T06:49:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 67.588863] postgres[1914]: [1914] ERROR: relation "table_without_models" does not exist1122gitlab # [ 67.590858] postgres[1914]: [1914] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1123gitlab # [ 69.754119] gitlab-workhorse[1306]: time="2026-07-09T06:49:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1124gitlab # [ 79.764102] gitlab-workhorse[1306]: time="2026-07-09T06:50:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 82.573767] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 82.577960] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 82.581662] systemd[1]: Starting gitlab.service...1128gitlab # [ 82.953780] bundle[2117]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 82.955871] bundle[2117]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 82.962163] bundle[2117]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 85.883623] sidekiq-cluster[2116]: {"severity":"INFO","time":"2026-07-09T06:50:14.668Z","message":"Starting cluster with 1 processes"}1132gitlab # [ 86.058518] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.839Z","pid":2117,"message":"* Enabling systemd notification integration"}1133gitlab # [ 86.061179] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"Puma starting in cluster mode..."}1134gitlab # [ 86.063884] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1135gitlab # [ 86.067318] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1136gitlab # [ 86.069779] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Min threads: 0"}1137gitlab # [ 86.073154] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Max threads: 4"}1138gitlab # [ 86.076287] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Environment: production"}1139gitlab # [ 86.078927] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Master PID: 2117"}1140gitlab # [ 86.082948] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Workers: 2"}1141gitlab # [ 86.086988] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1142gitlab # [ 86.089813] bundle[2117]: {"timestamp":"2026-07-09T06:50:14.861Z","pid":2117,"message":"* Preloading application"}1143gitlab # [ 89.773671] gitlab-workhorse[1306]: time="2026-07-09T06:50:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 99.775588] gitlab-workhorse[1306]: time="2026-07-09T06:50:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 109.777316] gitlab-workhorse[1306]: time="2026-07-09T06:50:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 111.030267] bundle[2117]: W, [2026-07-09T06:50:39.832713 #2117] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1147gitlab # [ 113.456834] sidekiq-cluster[2147]: 2026-07-09T06:50:42.259Z pid=2147 tid=u0b INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2147, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426627072}1148gitlab # [ 113.644758] sidekiq-cluster[2147]: W, [2026-07-09T06:50:42.447943 #2147] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1149gitlab # [ 113.683975] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:42.487Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1150gitlab # [ 115.714167] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.516Z","message":"GitLab reliable fetch activated!"}1151gitlab # [ 115.720633] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.516Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1152gitlab # [ 115.743263] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.546Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1153gitlab # [ 115.759239] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.562Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1154gitlab # [ 115.768116] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.568Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1155gitlab # [ 115.779922] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.583Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1156gitlab # [ 115.793271] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.596Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1157gitlab # [ 115.805160] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.608Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1158gitlab # [ 115.817517] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.620Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1159gitlab # [ 115.829417] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.632Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1160gitlab # [ 115.841593] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.644Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1161gitlab # [ 115.854344] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.657Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1162gitlab # [ 115.866564] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.669Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1163gitlab # [ 115.878221] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.681Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1164gitlab # [ 115.887178] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.690Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1165gitlab # [ 115.896184] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.699Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1166gitlab # [ 115.905238] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.708Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1167gitlab # [ 115.915482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.718Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1168gitlab # [ 115.924879] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.728Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1169gitlab # [ 115.935530] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.738Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1170gitlab # [ 115.945389] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.748Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1171gitlab # [ 115.956788] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.760Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1172gitlab # [ 115.968414] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.771Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1173gitlab # [ 115.979832] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.783Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1174gitlab # [ 115.990188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.793Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1175gitlab # [ 116.000625] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.803Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1176gitlab # [ 116.010180] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.813Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1177gitlab # [ 116.019767] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.823Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1178gitlab # [ 116.030367] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.833Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1179gitlab # [ 116.040527] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.843Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1180gitlab # [ 116.050806] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.854Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1181gitlab # [ 116.061510] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.864Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1182gitlab # [ 116.072178] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.875Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1183gitlab # [ 116.081600] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.884Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1184gitlab # [ 116.092639] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.895Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1185gitlab # [ 116.102533] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.905Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1186gitlab # [ 116.113141] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.916Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1187gitlab # [ 116.121594] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.924Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1188gitlab # [ 116.129802] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.933Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1189gitlab # [ 116.137789] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.941Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1190gitlab # [ 116.147221] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.950Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1191gitlab # [ 116.155130] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.958Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1192gitlab # [ 116.163143] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.966Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1193gitlab # [ 116.171561] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.974Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1194gitlab # [ 116.179928] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.983Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1195gitlab # [ 116.191083] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:44.994Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1196gitlab # [ 116.200398] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.003Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1197gitlab # [ 116.248535] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.051Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1198gitlab # [ 116.254490] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.057Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1199gitlab # [ 116.261420] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.064Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1200gitlab # [ 116.267606] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.071Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1201gitlab # [ 116.274319] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.077Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1202gitlab # [ 116.280742] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.084Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1203gitlab # [ 116.286645] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.090Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1204gitlab # [ 116.293166] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.096Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1205gitlab # [ 116.306382] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.109Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1206gitlab # [ 116.313394] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.116Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1207gitlab # [ 116.323054] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.126Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1208gitlab # [ 116.331092] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.134Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1209gitlab # [ 116.339468] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.142Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1210gitlab # [ 116.347662] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.151Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1211gitlab # [ 116.355606] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.158Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1212gitlab # [ 116.364575] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.167Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1213gitlab # [ 116.372657] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.176Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1214gitlab # [ 116.382275] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.185Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1215gitlab # [ 116.391911] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.195Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1216gitlab # [ 116.401393] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.204Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1217gitlab # [ 116.410190] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.213Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1218gitlab # [ 116.418744] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.222Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1219gitlab # [ 116.426700] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.230Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1220gitlab # [ 116.437182] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.240Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1221gitlab # [ 116.445714] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.249Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1222gitlab # [ 116.455205] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.258Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1223gitlab # [ 116.463587] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.266Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1224gitlab # [ 116.472582] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.275Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1225gitlab # [ 116.481194] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.284Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1226gitlab # [ 116.489939] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.293Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1227gitlab # [ 116.499354] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.302Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1228gitlab # [ 116.508369] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.311Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1229gitlab # [ 116.517994] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.321Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1230gitlab # [ 116.527207] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.330Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1231gitlab # [ 116.536367] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.339Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1232gitlab # [ 116.545191] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.348Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1233gitlab # [ 116.553632] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.356Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1234gitlab # [ 116.562605] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.365Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1235gitlab # [ 116.573377] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.376Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1236gitlab # [ 116.583570] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.386Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1237gitlab # [ 116.594226] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.397Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1238gitlab # [ 116.603176] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.406Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1239gitlab # [ 116.613620] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.416Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1240gitlab # [ 116.622334] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.425Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1241gitlab # [ 116.630125] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.433Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1242gitlab # [ 116.638137] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.441Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1243gitlab # [ 116.645593] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.448Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1244gitlab # [ 116.653201] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.456Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1245gitlab # [ 116.661700] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.465Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1246gitlab # [ 116.669216] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.472Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1247gitlab # [ 116.677273] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.480Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1248gitlab # [ 116.689891] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.493Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1249gitlab # [ 116.699877] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.503Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1250gitlab # [ 116.708651] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.511Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1251gitlab # [ 116.737431] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.539Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1252gitlab # [ 116.742127] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.545Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1253gitlab # [ 116.750335] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.553Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1254gitlab # [ 116.763558] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.566Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1255gitlab # [ 116.774253] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.577Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1256gitlab # [ 116.783390] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.586Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1257gitlab # [ 116.793183] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.596Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1258gitlab # [ 116.801951] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.605Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1259gitlab # [ 116.811188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.614Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1260gitlab # [ 116.821861] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.625Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1261gitlab # [ 116.830620] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:50:45.633Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1262gitlab # [ 119.787851] gitlab-workhorse[1306]: time="2026-07-09T06:50:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1263gitlab # [ 129.791135] gitlab-workhorse[1306]: time="2026-07-09T06:50:58Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1264gitlab # [ 137.793850] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.596Z","message":"Booted Rails 7.2.3 application in production environment"}1265gitlab # [ 137.797664] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.600Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1266gitlab # [ 137.801048] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.600Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1267gitlab # [ 137.803987] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.600Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1268gitlab # [ 137.807363] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:06.603Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1269gitlab # [ 139.171110] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:07.973Z","message":"Cleaning working queues"}1270gitlab # [ 139.179534] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:07.982Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b28303ae4f19620a80d00c18","created_at":"2026-07-09T06:50:10.713Z","trace_propagation_headers":{"sentry-trace":"abdbc09bb6024011b71866522215552f-29c5355ce1e2476f","baggage":"sentry-trace_id=abdbc09bb6024011b71866522215552f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5a5c57f4ba6b5f96e50fe7e79de8e58f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:50:10.724Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b28303ae4f19620a80d00c18: start","job_status":"start","queue_duration_s":57.258307,"scheduling_latency_s":57.25837}1271gitlab # [ 139.430626] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:08.232Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b28303ae4f19620a80d00c18","created_at":"2026-07-09T06:50:10.713Z","trace_propagation_headers":{"sentry-trace":"abdbc09bb6024011b71866522215552f-29c5355ce1e2476f","baggage":"sentry-trace_id=abdbc09bb6024011b71866522215552f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5a5c57f4ba6b5f96e50fe7e79de8e58f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:50:10.724Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b28303ae4f19620a80d00c18: done: 0.250506 sec","job_status":"done","queue_duration_s":57.258307,"scheduling_latency_s":57.25837,"redis_calls":30,"redis_duration_s":0.016214,"redis_read_bytes":410,"redis_write_bytes":3701,"redis_feature_flag_calls":19,"redis_feature_flag_duration_s":0.009991,"redis_feature_flag_read_bytes":400,"redis_feature_flag_write_bytes":2454,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004073,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00215,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.106646794,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.250506,"completed_at":"2026-07-09T06:51:08.232Z","load_balancing_strategy":"primary","db_duration_s":0.088455,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1272gitlab # [ 139.456556] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1273gitlab # [ 139.458248] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1274gitlab # [ 139.460013] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"! #\u003cThread:0x00007678ab45dfb8 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1275gitlab # [ 139.461365] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"! #\u003cThread:0x00007678ab6787f8 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1276gitlab # [ 139.463130] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.259Z","pid":2117,"message":"Use Ctrl-C to stop"}1277gitlab # [ 139.464599] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.260Z","pid":2117,"message":"Sending status to systemd every 1.0 sec"}1278gitlab # [ 139.863263] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.666Z","pid":2117,"message":"- Worker 0 (PID: 2666) booted in 0.34s, phase: 0"}1279gitlab # [ 139.904638] bundle[2117]: {"timestamp":"2026-07-09T06:51:08.707Z","pid":2117,"message":"- Worker 1 (PID: 2670) booted in 0.36s, phase: 0"}1280gitlab # [ 139.918220] systemd[1]: Started gitlab.service.1281gitlab # [ 139.923363] systemd[1]: Reached target Common target for all GitLab services..1282gitlab # [ 139.928452] systemd[1]: Starting setup-gitlab.service...1283gitlab # [ 139.946663] setup-gitlab-start[2706]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1284gitlab # [ 142.292589] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:51:11 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 22991285gitlab # [ 142.300946] setup-gitlab-start[2726]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1286gitlab # [ 142.310961] setup-gitlab-start[2727]: ++ jq -r .access_token1287gitlab # [ 143.104782] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:51:11 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 7851288gitlab # [ 143.109971] setup-gitlab-start[2706]: + BEARER=1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f2041289gitlab # [ 143.112326] setup-gitlab-start[2706]: + AUTH='Authorization: Bearer 1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204'1290gitlab # [ 143.114870] setup-gitlab-start[2706]: + curl -fs -X PUT -H 'Authorization: Bearer 1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1291gitlab # [ 144.148437] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:51:12 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 10151292gitlab # [ 144.159720] setup-gitlab-start[2706]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1293gitlab # [ 144.168236] runuser[2747]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1294gitlab # [ 144.541358] setup-gitlab-start[2748]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1295gitlab # [ 144.544176] setup-gitlab-start[2748]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1296gitlab # [ 144.546291] setup-gitlab-start[2748]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1297gitlab # [ 145.117970] setup-gitlab-start[2748]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1298gitlab # [ 145.126726] setup-gitlab-start[2748]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1299gitlab # [ 145.135221] setup-gitlab-start[2748]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1300gitlab # [ 152.944985] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"70a7aea24063a326f42e21f4","created_at":"2026-07-09T06:51:21.725Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f4f9d22debb3ce13bf038e8f294fbf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:51:21.743Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-70a7aea24063a326f42e21f4: start","job_status":"start","queue_duration_s":0.003035,"scheduling_latency_s":0.003105}1301gitlab # [ 153.113279] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"70a7aea24063a326f42e21f4","created_at":"2026-07-09T06:51:21.725Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f4f9d22debb3ce13bf038e8f294fbf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:51:21.743Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-70a7aea24063a326f42e21f4: done: 0.169404 sec","job_status":"done","queue_duration_s":0.003035,"scheduling_latency_s":0.003105,"redis_calls":10,"redis_duration_s":0.027127000000000002,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001902,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004191,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003399874,"duration_s":0.169404,"completed_at":"2026-07-09T06:51:21.915Z","load_balancing_strategy":"primary","db_duration_s":0.018193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 153.203677] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b0043a41667db9be91b7ce6","created_at":"2026-07-09T06:51:21.810Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c6722523f6c61b8686cb0f4c25b8a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:21.846Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b0043a41667db9be91b7ce6: start","job_status":"start","queue_duration_s":0.073002,"scheduling_latency_s":0.073042}1303gitlab # [ 153.226496] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.940Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1304gitlab # [ 153.232077] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:21.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b0043a41667db9be91b7ce6","created_at":"2026-07-09T06:51:21.810Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97c6722523f6c61b8686cb0f4c25b8a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:21.846Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b0043a41667db9be91b7ce6: done: 0.02798 sec","job_status":"done","queue_duration_s":0.073002,"scheduling_latency_s":0.073042,"redis_calls":10,"redis_duration_s":0.004849,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000952,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000496432,"duration_s":0.02798,"completed_at":"2026-07-09T06:51:21.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 153.268635] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"16e3ec86d6c96d1ce2441143","created_at":"2026-07-09T06:51:21.890Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e9989e2223533b3d0742ef3f5e5f014","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.021Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-16e3ec86d6c96d1ce2441143: start","job_status":"start","queue_duration_s":0.017393,"scheduling_latency_s":0.017457}1306gitlab # [ 153.414608] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"16e3ec86d6c96d1ce2441143","created_at":"2026-07-09T06:51:21.890Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e9989e2223533b3d0742ef3f5e5f014","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.021Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-16e3ec86d6c96d1ce2441143: done: 0.178395 sec","job_status":"done","queue_duration_s":0.017393,"scheduling_latency_s":0.017457,"redis_calls":13,"redis_duration_s":0.089392,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.020907,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.009493,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.058992,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03048097,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.178395,"completed_at":"2026-07-09T06:51:22.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015872,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1307gitlab # [ 153.505635] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4ad7defeb37b96ba168d54cd","created_at":"2026-07-09T06:51:22.038Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55ff693b59552cbcbeebbdecf622dadf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.047Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4ad7defeb37b96ba168d54cd: start","job_status":"start","queue_duration_s":0.261162,"scheduling_latency_s":0.261227}1308gitlab # [ 153.602534] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4ad7defeb37b96ba168d54cd","created_at":"2026-07-09T06:51:22.038Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55ff693b59552cbcbeebbdecf622dadf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.047Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4ad7defeb37b96ba168d54cd: done: 0.097427 sec","job_status":"done","queue_duration_s":0.261162,"scheduling_latency_s":0.261227,"redis_calls":12,"redis_duration_s":0.043005,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.028406,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000612089,"duration_s":0.097427,"completed_at":"2026-07-09T06:51:22.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 153.652415] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5f51e811fd3cb041cf5391d8","created_at":"2026-07-09T06:51:22.064Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d942e171022ae1869fb5b03b95f4196","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.125Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5f51e811fd3cb041cf5391d8: start","job_status":"start","queue_duration_s":0.286806,"scheduling_latency_s":0.286867}1310gitlab # [ 153.664697] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5f51e811fd3cb041cf5391d8","created_at":"2026-07-09T06:51:22.064Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d942e171022ae1869fb5b03b95f4196","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.125Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5f51e811fd3cb041cf5391d8: done: 0.055716 sec","job_status":"done","queue_duration_s":0.286806,"scheduling_latency_s":0.286867,"redis_calls":12,"redis_duration_s":0.003893,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000715,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001125004,"exclusive_lock_hold_duration_s":0.005300394,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.055716,"completed_at":"2026-07-09T06:51:22.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 153.734275] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:22.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2cad8cfdb593de9f7688bfed","created_at":"2026-07-09T06:51:22.162Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65186eed3a203f686f4317392c151bab","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.180Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2cad8cfdb593de9f7688bfed: start","job_status":"start","queue_duration_s":0.289366,"scheduling_latency_s":0.289386}1312gitlab # [ 154.234272] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"2cad8cfdb593de9f7688bfed","created_at":"2026-07-09T06:51:22.162Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65186eed3a203f686f4317392c151bab","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.180Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-2cad8cfdb593de9f7688bfed: done: 0.567592 sec","job_status":"done","queue_duration_s":0.289366,"scheduling_latency_s":0.289386,"redis_calls":574,"redis_duration_s":0.24959599999999998,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.247695,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.357253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358426,"duration_s":0.567592,"completed_at":"2026-07-09T06:51:23.036Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 154.339195] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.142Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cfb31c9f7612c04a003c09e1","created_at":"2026-07-09T06:51:22.201Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b3e095e33be9fe5a140c3761a0f9ab0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.220Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cfb31c9f7612c04a003c09e1: start","job_status":"start","queue_duration_s":0.922182,"scheduling_latency_s":0.922219}1314gitlab # [ 154.401558] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cfb31c9f7612c04a003c09e1","created_at":"2026-07-09T06:51:22.201Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b3e095e33be9fe5a140c3761a0f9ab0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.220Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cfb31c9f7612c04a003c09e1: done: 0.062114 sec","job_status":"done","queue_duration_s":0.922182,"scheduling_latency_s":0.922219,"redis_calls":12,"redis_duration_s":0.004444,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002449,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000996495,"exclusive_lock_hold_duration_s":0.011014528,"duration_s":0.062114,"completed_at":"2026-07-09T06:51:23.204Z","load_balancing_strategy":"primary","db_duration_s":0.005412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 154.438081] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e8edd68e2f315f9fabbe4d8c","created_at":"2026-07-09T06:51:22.319Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c84e5c3bfb13a87145eafa37340e4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:51:22.345Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e8edd68e2f315f9fabbe4d8c: start","job_status":"start","queue_duration_s":0.862396,"scheduling_latency_s":0.862445}1316gitlab # [ 154.439242] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.231Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1317gitlab # [ 154.440092] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e8edd68e2f315f9fabbe4d8c","created_at":"2026-07-09T06:51:22.319Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c84e5c3bfb13a87145eafa37340e4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:51:22.345Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e8edd68e2f315f9fabbe4d8c: done: 0.031173 sec","job_status":"done","queue_duration_s":0.862396,"scheduling_latency_s":0.862445,"redis_calls":10,"redis_duration_s":0.005026,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001199,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000600914,"duration_s":0.031173,"completed_at":"2026-07-09T06:51:23.238Z","load_balancing_strategy":"primary","db_duration_s":0.000935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 154.555668] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8d6fd29b3ff90f7ac5a8a8a9","created_at":"2026-07-09T06:51:22.386Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.399Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8d6fd29b3ff90f7ac5a8a8a9: start","job_status":"start","queue_duration_s":0.959363,"scheduling_latency_s":0.959401}1319gitlab # [ 154.719932] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8d6fd29b3ff90f7ac5a8a8a9","created_at":"2026-07-09T06:51:22.386Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.399Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8d6fd29b3ff90f7ac5a8a8a9: done: 0.164311 sec","job_status":"done","queue_duration_s":0.959363,"scheduling_latency_s":0.959401,"redis_calls":71,"redis_duration_s":0.034127,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004633,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001596,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.02658,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001318,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.104554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000667683,"duration_s":0.164311,"completed_at":"2026-07-09T06:51:23.522Z","load_balancing_strategy":"primary","db_duration_s":0.003897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 154.825459] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:23.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3625be359b6b2aebb3dd8958","created_at":"2026-07-09T06:51:22.411Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da756f5492c3f47b127605fda5e474ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3625be359b6b2aebb3dd8958: start","job_status":"start","queue_duration_s":1.029522,"scheduling_latency_s":1.029585}1321gitlab # [ 155.272795] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3625be359b6b2aebb3dd8958","created_at":"2026-07-09T06:51:22.411Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da756f5492c3f47b127605fda5e474ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3625be359b6b2aebb3dd8958: done: 0.446971 sec","job_status":"done","queue_duration_s":1.029522,"scheduling_latency_s":1.029585,"redis_calls":14,"redis_duration_s":0.008429,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002559,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.055,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.340636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001403251,"exclusive_lock_hold_duration_s":0.40073082000000004,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.446971,"completed_at":"2026-07-09T06:51:24.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 155.370689] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"089f7e50bd695d7739f1be5f","created_at":"2026-07-09T06:51:22.606Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9f865d7ee5cef95c619d9d51cc488fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.615Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-089f7e50bd695d7739f1be5f: start","job_status":"start","queue_duration_s":1.558499,"scheduling_latency_s":1.558539}1323gitlab # [ 155.451449] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"089f7e50bd695d7739f1be5f","created_at":"2026-07-09T06:51:22.606Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9f865d7ee5cef95c619d9d51cc488fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.615Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-089f7e50bd695d7739f1be5f: done: 0.080337 sec","job_status":"done","queue_duration_s":1.558499,"scheduling_latency_s":1.558539,"redis_calls":12,"redis_duration_s":0.0060609999999999995,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000845,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002257,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000775238,"exclusive_lock_hold_duration_s":0.028949769,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.080337,"completed_at":"2026-07-09T06:51:24.253Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 155.535385] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"91b99ca81f7852a09c971f55","created_at":"2026-07-09T06:51:22.630Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c345a84abfbbb1d3dc8159fd7b32a94a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.639Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-91b99ca81f7852a09c971f55: start","job_status":"start","queue_duration_s":1.698439,"scheduling_latency_s":1.698504}1325gitlab # [ 155.581304] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.384Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1326gitlab # [ 155.599497] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"91b99ca81f7852a09c971f55","created_at":"2026-07-09T06:51:22.630Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c345a84abfbbb1d3dc8159fd7b32a94a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.639Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-91b99ca81f7852a09c971f55: done: 0.064371 sec","job_status":"done","queue_duration_s":1.698439,"scheduling_latency_s":1.698504,"redis_calls":10,"redis_duration_s":0.007035,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000869,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000682768,"duration_s":0.064371,"completed_at":"2026-07-09T06:51:24.402Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 155.668376] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.471Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1216b3e01fbd020c0d7834e6","created_at":"2026-07-09T06:51:22.650Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38eac13f6572bdf71eaee4a87d4f1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.657Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1216b3e01fbd020c0d7834e6: start","job_status":"start","queue_duration_s":1.813662,"scheduling_latency_s":1.813724}1328gitlab # [ 155.715225] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"1216b3e01fbd020c0d7834e6","created_at":"2026-07-09T06:51:22.650Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d38eac13f6572bdf71eaee4a87d4f1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.657Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-1216b3e01fbd020c0d7834e6: done: 0.046539 sec","job_status":"done","queue_duration_s":1.813662,"scheduling_latency_s":1.813724,"redis_calls":10,"redis_duration_s":0.005738,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001117,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000940064,"duration_s":0.046539,"completed_at":"2026-07-09T06:51:24.517Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 155.785288] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6680868bb36c4c1f7998acc9","created_at":"2026-07-09T06:51:22.665Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"049cff5f35e1590950627fcbadc149d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.671Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6680868bb36c4c1f7998acc9: start","job_status":"start","queue_duration_s":1.916542,"scheduling_latency_s":1.916607}1330gitlab # [ 155.814556] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.617Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1331gitlab # [ 155.830203] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6680868bb36c4c1f7998acc9","created_at":"2026-07-09T06:51:22.665Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"049cff5f35e1590950627fcbadc149d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.671Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6680868bb36c4c1f7998acc9: done: 0.045028 sec","job_status":"done","queue_duration_s":1.916542,"scheduling_latency_s":1.916607,"redis_calls":10,"redis_duration_s":0.006300999999999999,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001316,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000922185,"duration_s":0.045028,"completed_at":"2026-07-09T06:51:24.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 155.833497] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a86cba595c3feb0447bae7dd","created_at":"2026-07-09T06:51:22.678Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7470f8bd5c246cddd215dbc2b8bc6b93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:51:22.685Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a86cba595c3feb0447bae7dd: start","job_status":"start","queue_duration_s":1.951055,"scheduling_latency_s":1.951094}1333gitlab # [ 155.924982] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.728Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1334gitlab # [ 155.938954] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a86cba595c3feb0447bae7dd","created_at":"2026-07-09T06:51:22.678Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7470f8bd5c246cddd215dbc2b8bc6b93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:51:22.685Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a86cba595c3feb0447bae7dd: done: 0.104756 sec","job_status":"done","queue_duration_s":1.951055,"scheduling_latency_s":1.951094,"redis_calls":10,"redis_duration_s":0.004847,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001192,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.072,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000948165,"duration_s":0.104756,"completed_at":"2026-07-09T06:51:24.741Z","load_balancing_strategy":"primary","db_duration_s":0.072147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 156.006130] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"71ef612bb6e72fa4f7865a67","created_at":"2026-07-09T06:51:22.699Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569a577e56c5f49c21124b85e6c2684a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.712Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-71ef612bb6e72fa4f7865a67: start","job_status":"start","queue_duration_s":2.09672,"scheduling_latency_s":2.096783}1336gitlab # [ 156.108617] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.911Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"71ef612bb6e72fa4f7865a67","created_at":"2026-07-09T06:51:22.699Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569a577e56c5f49c21124b85e6c2684a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.712Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-71ef612bb6e72fa4f7865a67: done: 0.102641 sec","job_status":"done","queue_duration_s":2.09672,"scheduling_latency_s":2.096783,"redis_calls":16,"redis_duration_s":0.008469,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004132,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000997892,"duration_s":0.102641,"completed_at":"2026-07-09T06:51:24.911Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 156.113136] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.915Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bcdfcb94d118b8031640a1cb","created_at":"2026-07-09T06:51:22.722Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.731Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bcdfcb94d118b8031640a1cb: start","job_status":"start","queue_duration_s":2.183705,"scheduling_latency_s":2.183757}1338gitlab # [ 156.213486] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bcdfcb94d118b8031640a1cb","created_at":"2026-07-09T06:51:22.722Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.731Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bcdfcb94d118b8031640a1cb: done: 0.039238 sec","job_status":"done","queue_duration_s":2.183705,"scheduling_latency_s":2.183757,"redis_calls":19,"redis_duration_s":0.005241,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001051,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000399,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002675,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000439441,"duration_s":0.039238,"completed_at":"2026-07-09T06:51:24.954Z","load_balancing_strategy":"primary","db_duration_s":0.00469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 156.218968] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6e9e02254cfdfe14c32b3a1","created_at":"2026-07-09T06:51:22.740Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"404265cf8c38963254e52bb15fcb58ae","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.752Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6e9e02254cfdfe14c32b3a1: start","job_status":"start","queue_duration_s":2.2037,"scheduling_latency_s":2.20372}1340gitlab # [ 156.235602] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6e9e02254cfdfe14c32b3a1","created_at":"2026-07-09T06:51:22.740Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"404265cf8c38963254e52bb15fcb58ae","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.752Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6e9e02254cfdfe14c32b3a1: done: 0.022765 sec","job_status":"done","queue_duration_s":2.2037,"scheduling_latency_s":2.20372,"redis_calls":10,"redis_duration_s":0.003487,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000701,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000484978,"duration_s":0.022765,"completed_at":"2026-07-09T06:51:24.978Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 156.315522] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ba281a9fb713ff50542480af","created_at":"2026-07-09T06:51:22.767Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01822331cc054980c8ce36f766af0a16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.776Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ba281a9fb713ff50542480af: start","job_status":"start","queue_duration_s":2.204081,"scheduling_latency_s":2.204101}1342gitlab # [ 156.321369] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.992Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1343gitlab # [ 156.392596] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:24.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ba281a9fb713ff50542480af","created_at":"2026-07-09T06:51:22.767Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01822331cc054980c8ce36f766af0a16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.776Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ba281a9fb713ff50542480af: done: 0.018756 sec","job_status":"done","queue_duration_s":2.204081,"scheduling_latency_s":2.204101,"redis_calls":10,"redis_duration_s":0.003618,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000751,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000486933,"duration_s":0.018756,"completed_at":"2026-07-09T06:51:24.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 156.394874] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4c4ee8f1738e8dcc045f043a","created_at":"2026-07-09T06:51:22.785Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fde370cecc046592443ddc5e7dc7660e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.796Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4c4ee8f1738e8dcc045f043a: start","job_status":"start","queue_duration_s":2.205152,"scheduling_latency_s":2.205172}1345gitlab # [ 156.464286] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"4c4ee8f1738e8dcc045f043a","created_at":"2026-07-09T06:51:22.785Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fde370cecc046592443ddc5e7dc7660e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.796Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-4c4ee8f1738e8dcc045f043a: done: 0.046923 sec","job_status":"done","queue_duration_s":2.205152,"scheduling_latency_s":2.205172,"redis_calls":12,"redis_duration_s":0.004666,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000872,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057214,"exclusive_lock_hold_duration_s":0.016844319,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.046923,"completed_at":"2026-07-09T06:51:25.048Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 156.483891] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e68726b229cf9d3598ee85a1","created_at":"2026-07-09T06:51:22.808Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ba6187a45160e2eb4922b243fa9de7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.819Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e68726b229cf9d3598ee85a1: start","job_status":"start","queue_duration_s":2.230288,"scheduling_latency_s":2.230308}1347gitlab # [ 156.553280] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e68726b229cf9d3598ee85a1","created_at":"2026-07-09T06:51:22.808Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ba6187a45160e2eb4922b243fa9de7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:51:22.819Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e68726b229cf9d3598ee85a1: done: 0.093505 sec","job_status":"done","queue_duration_s":2.230288,"scheduling_latency_s":2.230308,"redis_calls":10,"redis_duration_s":0.004342,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000994,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000988114,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.093505,"completed_at":"2026-07-09T06:51:25.143Z","load_balancing_strategy":"primary","db_duration_s":0.021208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 156.624921] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a452b5508fb5325765b9cc8a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a452b5508fb5325765b9cc8a: start","job_status":"start","queue_duration_s":1.632244,"scheduling_latency_s":1.632265}1349gitlab # [ 156.626339] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a452b5508fb5325765b9cc8a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a452b5508fb5325765b9cc8a: done: 0.026399 sec","job_status":"done","queue_duration_s":1.632244,"scheduling_latency_s":1.632265,"redis_calls":16,"redis_duration_s":0.004275,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0022,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000716851,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026399,"completed_at":"2026-07-09T06:51:25.171Z","load_balancing_strategy":"primary","db_duration_s":0.000629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 156.698750] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b65a1edfd91a32d99e0629","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b65a1edfd91a32d99e0629: start","job_status":"start","queue_duration_s":1.660597,"scheduling_latency_s":1.660619}1351gitlab # [ 156.700188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b65a1edfd91a32d99e0629","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b65a1edfd91a32d99e0629: done: 0.015714 sec","job_status":"done","queue_duration_s":1.660597,"scheduling_latency_s":1.660619,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015714,"completed_at":"2026-07-09T06:51:25.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 156.771852] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ef833a0b5fb0ad207b4443d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ef833a0b5fb0ad207b4443d: start","job_status":"start","queue_duration_s":1.677581,"scheduling_latency_s":1.677601}1353gitlab # [ 156.773194] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ef833a0b5fb0ad207b4443d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ef833a0b5fb0ad207b4443d: done: 0.014916 sec","job_status":"done","queue_duration_s":1.677581,"scheduling_latency_s":1.677601,"redis_calls":13,"redis_duration_s":0.003069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014916,"completed_at":"2026-07-09T06:51:25.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 156.843818] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61bab904e1972e018f3234ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61bab904e1972e018f3234ff: start","job_status":"start","queue_duration_s":1.694039,"scheduling_latency_s":1.694059}1355gitlab # [ 156.845200] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61bab904e1972e018f3234ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61bab904e1972e018f3234ff: done: 0.014932 sec","job_status":"done","queue_duration_s":1.694039,"scheduling_latency_s":1.694059,"redis_calls":13,"redis_duration_s":0.0031060000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014932,"completed_at":"2026-07-09T06:51:25.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 156.937966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e7f9159d150af9e2d9ba2b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e7f9159d150af9e2d9ba2b: start","job_status":"start","queue_duration_s":1.710317,"scheduling_latency_s":1.710337}1357gitlab # [ 156.939053] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e7f9159d150af9e2d9ba2b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e7f9159d150af9e2d9ba2b: done: 0.015735 sec","job_status":"done","queue_duration_s":1.710317,"scheduling_latency_s":1.710337,"redis_calls":13,"redis_duration_s":0.003664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002694,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015735,"completed_at":"2026-07-09T06:51:25.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 157.018160] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4febc853da917a8b0af44316","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4febc853da917a8b0af44316: start","job_status":"start","queue_duration_s":1.727319,"scheduling_latency_s":1.727339}1359gitlab # [ 157.019512] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4febc853da917a8b0af44316","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4febc853da917a8b0af44316: done: 0.015117 sec","job_status":"done","queue_duration_s":1.727319,"scheduling_latency_s":1.727339,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015117,"completed_at":"2026-07-09T06:51:25.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 157.090200] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac2dc358859fe81e0cea03e9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac2dc358859fe81e0cea03e9: start","job_status":"start","queue_duration_s":1.743725,"scheduling_latency_s":1.743745}1361gitlab # [ 157.091780] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac2dc358859fe81e0cea03e9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac2dc358859fe81e0cea03e9: done: 0.014978 sec","job_status":"done","queue_duration_s":1.743725,"scheduling_latency_s":1.743745,"redis_calls":13,"redis_duration_s":0.002963,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014978,"completed_at":"2026-07-09T06:51:25.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 157.165271] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"778c62d0a1c144615bc63a18","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-778c62d0a1c144615bc63a18: start","job_status":"start","queue_duration_s":1.760196,"scheduling_latency_s":1.76022}1363gitlab # [ 157.165972] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"778c62d0a1c144615bc63a18","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-778c62d0a1c144615bc63a18: done: 0.015225 sec","job_status":"done","queue_duration_s":1.760196,"scheduling_latency_s":1.76022,"redis_calls":13,"redis_duration_s":0.003043,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015225,"completed_at":"2026-07-09T06:51:25.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 157.249817] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd9276541a5766f9ae2b774f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd9276541a5766f9ae2b774f: start","job_status":"start","queue_duration_s":1.77652,"scheduling_latency_s":1.77654}1365gitlab # [ 157.251195] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd9276541a5766f9ae2b774f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd9276541a5766f9ae2b774f: done: 0.014782 sec","job_status":"done","queue_duration_s":1.77652,"scheduling_latency_s":1.77654,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014782,"completed_at":"2026-07-09T06:51:25.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 157.321286] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fb91c341766f35a48b9a41e","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fb91c341766f35a48b9a41e: start","job_status":"start","queue_duration_s":1.792557,"scheduling_latency_s":1.792577}1367gitlab # [ 157.322826] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fb91c341766f35a48b9a41e","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fb91c341766f35a48b9a41e: done: 0.014926 sec","job_status":"done","queue_duration_s":1.792557,"scheduling_latency_s":1.792577,"redis_calls":13,"redis_duration_s":0.003015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014926,"completed_at":"2026-07-09T06:51:25.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 157.418278] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bdedc38d17382c12b87bc56","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bdedc38d17382c12b87bc56: start","job_status":"start","queue_duration_s":1.808758,"scheduling_latency_s":1.808777}1369gitlab # [ 157.419391] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bdedc38d17382c12b87bc56","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bdedc38d17382c12b87bc56: done: 0.014953 sec","job_status":"done","queue_duration_s":1.808758,"scheduling_latency_s":1.808777,"redis_calls":13,"redis_duration_s":0.0027949999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014953,"completed_at":"2026-07-09T06:51:25.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 157.486424] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"139d66e0154b91f7ea8d4fb4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-139d66e0154b91f7ea8d4fb4: start","job_status":"start","queue_duration_s":1.824962,"scheduling_latency_s":1.824983}1371gitlab # [ 157.487687] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"139d66e0154b91f7ea8d4fb4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-139d66e0154b91f7ea8d4fb4: done: 0.015078 sec","job_status":"done","queue_duration_s":1.824962,"scheduling_latency_s":1.824983,"redis_calls":13,"redis_duration_s":0.002727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015078,"completed_at":"2026-07-09T06:51:25.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 157.561093] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e75d8f5804797e63b05034","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e75d8f5804797e63b05034: start","job_status":"start","queue_duration_s":1.841585,"scheduling_latency_s":1.841605}1373gitlab # [ 157.562393] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e75d8f5804797e63b05034","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e75d8f5804797e63b05034: done: 0.014915 sec","job_status":"done","queue_duration_s":1.841585,"scheduling_latency_s":1.841605,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014915,"completed_at":"2026-07-09T06:51:25.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 157.631983] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d75236282d60a2d8b0c12467","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d75236282d60a2d8b0c12467: start","job_status":"start","queue_duration_s":1.857762,"scheduling_latency_s":1.857781}1375gitlab # [ 157.633179] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d75236282d60a2d8b0c12467","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d75236282d60a2d8b0c12467: done: 0.015301 sec","job_status":"done","queue_duration_s":1.857762,"scheduling_latency_s":1.857781,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015301,"completed_at":"2026-07-09T06:51:25.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 157.731096] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92923f737911224179dfcd0d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92923f737911224179dfcd0d: start","job_status":"start","queue_duration_s":1.874293,"scheduling_latency_s":1.874313}1377gitlab # [ 157.732386] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92923f737911224179dfcd0d","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92923f737911224179dfcd0d: done: 0.01498 sec","job_status":"done","queue_duration_s":1.874293,"scheduling_latency_s":1.874313,"redis_calls":13,"redis_duration_s":0.0027919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-07-09T06:51:25.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 157.809235] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65ba9341a481e9321d87b3b6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65ba9341a481e9321d87b3b6: start","job_status":"start","queue_duration_s":1.890553,"scheduling_latency_s":1.890572}1379gitlab # [ 157.810299] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65ba9341a481e9321d87b3b6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65ba9341a481e9321d87b3b6: done: 0.015209 sec","job_status":"done","queue_duration_s":1.890553,"scheduling_latency_s":1.890572,"redis_calls":13,"redis_duration_s":0.003028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015209,"completed_at":"2026-07-09T06:51:25.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 157.885271] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a98fab433ac2bc8bee18dfb9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a98fab433ac2bc8bee18dfb9: start","job_status":"start","queue_duration_s":1.907387,"scheduling_latency_s":1.907408}1381gitlab # [ 157.886879] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a98fab433ac2bc8bee18dfb9","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a98fab433ac2bc8bee18dfb9: done: 0.01556 sec","job_status":"done","queue_duration_s":1.907387,"scheduling_latency_s":1.907408,"redis_calls":13,"redis_duration_s":0.003041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01556,"completed_at":"2026-07-09T06:51:25.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 157.976399] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e31179a52e9ef3f551b70feb","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e31179a52e9ef3f551b70feb: start","job_status":"start","queue_duration_s":1.92428,"scheduling_latency_s":1.9243}1383gitlab # [ 157.978261] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e31179a52e9ef3f551b70feb","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e31179a52e9ef3f551b70feb: done: 0.014976 sec","job_status":"done","queue_duration_s":1.92428,"scheduling_latency_s":1.9243,"redis_calls":13,"redis_duration_s":0.002792,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014976,"completed_at":"2026-07-09T06:51:25.452Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 158.045993] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a97498bb491a8ef4cdc2748","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a97498bb491a8ef4cdc2748: start","job_status":"start","queue_duration_s":1.940558,"scheduling_latency_s":1.940579}1385gitlab # [ 158.047262] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a97498bb491a8ef4cdc2748","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a97498bb491a8ef4cdc2748: done: 0.014885 sec","job_status":"done","queue_duration_s":1.940558,"scheduling_latency_s":1.940579,"redis_calls":13,"redis_duration_s":0.0027790000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014885,"completed_at":"2026-07-09T06:51:25.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 158.117376] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8518462eb71ded0fd5dd82c4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8518462eb71ded0fd5dd82c4: start","job_status":"start","queue_duration_s":1.956731,"scheduling_latency_s":1.956751}1387gitlab # [ 158.118966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8518462eb71ded0fd5dd82c4","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8518462eb71ded0fd5dd82c4: done: 0.015424 sec","job_status":"done","queue_duration_s":1.956731,"scheduling_latency_s":1.956751,"redis_calls":13,"redis_duration_s":0.0027930000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015424,"completed_at":"2026-07-09T06:51:25.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 158.189764] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc5a9c2f1f76dde782f205c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc5a9c2f1f76dde782f205c1: start","job_status":"start","queue_duration_s":1.973399,"scheduling_latency_s":1.973419}1389gitlab # [ 158.191193] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc5a9c2f1f76dde782f205c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc5a9c2f1f76dde782f205c1: done: 0.015361 sec","job_status":"done","queue_duration_s":1.973399,"scheduling_latency_s":1.973419,"redis_calls":13,"redis_duration_s":0.0027140000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015361,"completed_at":"2026-07-09T06:51:25.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 158.287269] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7de8028e9e2c439349d314","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7de8028e9e2c439349d314: start","job_status":"start","queue_duration_s":1.990112,"scheduling_latency_s":1.990133}1391gitlab # [ 158.288843] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7de8028e9e2c439349d314","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7de8028e9e2c439349d314: done: 0.015702 sec","job_status":"done","queue_duration_s":1.990112,"scheduling_latency_s":1.990133,"redis_calls":13,"redis_duration_s":0.002993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015702,"completed_at":"2026-07-09T06:51:25.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 158.308256] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0eeebacaebdca8f92885f0a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0eeebacaebdca8f92885f0a: start","job_status":"start","queue_duration_s":2.00706,"scheduling_latency_s":2.00708}1393gitlab # [ 158.378708] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0eeebacaebdca8f92885f0a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0eeebacaebdca8f92885f0a: done: 0.015168 sec","job_status":"done","queue_duration_s":2.00706,"scheduling_latency_s":2.00708,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015168,"completed_at":"2026-07-09T06:51:25.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 158.386731] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06a1fa71cbf1bb73341667d8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06a1fa71cbf1bb73341667d8: start","job_status":"start","queue_duration_s":2.023898,"scheduling_latency_s":2.023918}1395gitlab # [ 158.457630] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06a1fa71cbf1bb73341667d8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06a1fa71cbf1bb73341667d8: done: 0.018317 sec","job_status":"done","queue_duration_s":2.023898,"scheduling_latency_s":2.023918,"redis_calls":13,"redis_duration_s":0.0031810000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018317,"completed_at":"2026-07-09T06:51:25.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 158.465413] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a3dfc692afe8a89994140b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a3dfc692afe8a89994140b: start","job_status":"start","queue_duration_s":2.043804,"scheduling_latency_s":2.04383}1397gitlab # [ 158.538355] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a3dfc692afe8a89994140b","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a3dfc692afe8a89994140b: done: 0.015934 sec","job_status":"done","queue_duration_s":2.043804,"scheduling_latency_s":2.04383,"redis_calls":13,"redis_duration_s":0.002703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015934,"completed_at":"2026-07-09T06:51:25.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 158.545077] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec64d77e029bfc782237a852","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec64d77e029bfc782237a852: start","job_status":"start","queue_duration_s":2.061262,"scheduling_latency_s":2.061289}1399gitlab # [ 158.614520] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec64d77e029bfc782237a852","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec64d77e029bfc782237a852: done: 0.016831 sec","job_status":"done","queue_duration_s":2.061262,"scheduling_latency_s":2.061289,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016831,"completed_at":"2026-07-09T06:51:25.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 158.682482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d93ab61c9b48b89a865fe064","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d93ab61c9b48b89a865fe064: start","job_status":"start","queue_duration_s":2.079355,"scheduling_latency_s":2.079385}1401gitlab # [ 158.683708] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d93ab61c9b48b89a865fe064","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d93ab61c9b48b89a865fe064: done: 0.015032 sec","job_status":"done","queue_duration_s":2.079355,"scheduling_latency_s":2.079385,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015032,"completed_at":"2026-07-09T06:51:25.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 158.702938] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81ee21e1764801bb0da3966a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81ee21e1764801bb0da3966a: start","job_status":"start","queue_duration_s":2.096069,"scheduling_latency_s":2.09609}1403gitlab # [ 158.773386] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81ee21e1764801bb0da3966a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81ee21e1764801bb0da3966a: done: 0.01498 sec","job_status":"done","queue_duration_s":2.096069,"scheduling_latency_s":2.09609,"redis_calls":13,"redis_duration_s":0.002771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-07-09T06:51:25.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 158.844110] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cd271a1b9315d33cab427a8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cd271a1b9315d33cab427a8: start","job_status":"start","queue_duration_s":2.11226,"scheduling_latency_s":2.11228}1405gitlab # [ 158.845413] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cd271a1b9315d33cab427a8","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cd271a1b9315d33cab427a8: done: 0.014936 sec","job_status":"done","queue_duration_s":2.11226,"scheduling_latency_s":2.11228,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014936,"completed_at":"2026-07-09T06:51:25.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 158.914199] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"430f7a73b18aefd4c2d9b585","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-430f7a73b18aefd4c2d9b585: start","job_status":"start","queue_duration_s":2.128437,"scheduling_latency_s":2.128457}1407gitlab # [ 158.915802] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"430f7a73b18aefd4c2d9b585","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-430f7a73b18aefd4c2d9b585: done: 0.015221 sec","job_status":"done","queue_duration_s":2.128437,"scheduling_latency_s":2.128457,"redis_calls":13,"redis_duration_s":0.002996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015221,"completed_at":"2026-07-09T06:51:25.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 158.985252] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67d6e4b4dcc14be0b006e074","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67d6e4b4dcc14be0b006e074: start","job_status":"start","queue_duration_s":2.145044,"scheduling_latency_s":2.145063}1409gitlab # [ 158.986798] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67d6e4b4dcc14be0b006e074","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67d6e4b4dcc14be0b006e074: done: 0.015682 sec","job_status":"done","queue_duration_s":2.145044,"scheduling_latency_s":2.145063,"redis_calls":13,"redis_duration_s":0.0029890000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015682,"completed_at":"2026-07-09T06:51:25.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 159.061206] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30a8876d41e27c2ddb88784f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30a8876d41e27c2ddb88784f: start","job_status":"start","queue_duration_s":2.162145,"scheduling_latency_s":2.162165}1411gitlab # [ 159.062884] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30a8876d41e27c2ddb88784f","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30a8876d41e27c2ddb88784f: done: 0.015349 sec","job_status":"done","queue_duration_s":2.162145,"scheduling_latency_s":2.162165,"redis_calls":13,"redis_duration_s":0.002902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015349,"completed_at":"2026-07-09T06:51:25.690Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 159.160319] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60bd15f4ca420c51f31241ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60bd15f4ca420c51f31241ff: start","job_status":"start","queue_duration_s":2.178761,"scheduling_latency_s":2.178781}1413gitlab # [ 159.161889] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60bd15f4ca420c51f31241ff","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60bd15f4ca420c51f31241ff: done: 0.229893 sec","job_status":"done","queue_duration_s":2.178761,"scheduling_latency_s":2.178781,"redis_calls":13,"redis_duration_s":0.0031409999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.229893,"completed_at":"2026-07-09T06:51:25.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 159.235865] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15d4d88d092aab04ba0d61c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15d4d88d092aab04ba0d61c1: start","job_status":"start","queue_duration_s":2.410255,"scheduling_latency_s":2.410282}1415gitlab # [ 159.237251] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15d4d88d092aab04ba0d61c1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15d4d88d092aab04ba0d61c1: done: 0.016505 sec","job_status":"done","queue_duration_s":2.410255,"scheduling_latency_s":2.410282,"redis_calls":13,"redis_duration_s":0.00273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016505,"completed_at":"2026-07-09T06:51:25.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 159.306764] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36f3d3419b51a3f829636dc5","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36f3d3419b51a3f829636dc5: start","job_status":"start","queue_duration_s":2.428066,"scheduling_latency_s":2.428086}1417gitlab # [ 159.308142] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36f3d3419b51a3f829636dc5","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36f3d3419b51a3f829636dc5: done: 0.014907 sec","job_status":"done","queue_duration_s":2.428066,"scheduling_latency_s":2.428086,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014907,"completed_at":"2026-07-09T06:51:25.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 159.385488] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e441b4aa605a6758af3eade","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e441b4aa605a6758af3eade: start","job_status":"start","queue_duration_s":2.44422,"scheduling_latency_s":2.44424}1419gitlab # [ 159.386979] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e441b4aa605a6758af3eade","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e441b4aa605a6758af3eade: done: 0.015078 sec","job_status":"done","queue_duration_s":2.44422,"scheduling_latency_s":2.44424,"redis_calls":13,"redis_duration_s":0.002966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015078,"completed_at":"2026-07-09T06:51:25.972Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 159.458248] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec7523cc93b0ae16b72ed73","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec7523cc93b0ae16b72ed73: start","job_status":"start","queue_duration_s":2.460633,"scheduling_latency_s":2.460654}1421gitlab # [ 159.459386] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec7523cc93b0ae16b72ed73","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec7523cc93b0ae16b72ed73: done: 0.01517 sec","job_status":"done","queue_duration_s":2.460633,"scheduling_latency_s":2.460654,"redis_calls":13,"redis_duration_s":0.0028450000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01517,"completed_at":"2026-07-09T06:51:25.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 159.538225] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:25.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96a7887f853526ccf965a016","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96a7887f853526ccf965a016: start","job_status":"start","queue_duration_s":2.477047,"scheduling_latency_s":2.477067}1423gitlab # [ 159.539761] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96a7887f853526ccf965a016","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96a7887f853526ccf965a016: done: 0.015093 sec","job_status":"done","queue_duration_s":2.477047,"scheduling_latency_s":2.477067,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015093,"completed_at":"2026-07-09T06:51:26.005Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 159.620122] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ac03274ab740748918cd06","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ac03274ab740748918cd06: start","job_status":"start","queue_duration_s":2.494166,"scheduling_latency_s":2.494204}1425gitlab # [ 159.621493] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15ac03274ab740748918cd06","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15ac03274ab740748918cd06: done: 0.018292 sec","job_status":"done","queue_duration_s":2.494166,"scheduling_latency_s":2.494204,"redis_calls":13,"redis_duration_s":0.003144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018292,"completed_at":"2026-07-09T06:51:26.025Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 159.692265] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e0e2468602b80fb6e1dff1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e0e2468602b80fb6e1dff1: start","job_status":"start","queue_duration_s":2.513714,"scheduling_latency_s":2.513735}1427gitlab # [ 159.693966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e0e2468602b80fb6e1dff1","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e0e2468602b80fb6e1dff1: done: 0.014778 sec","job_status":"done","queue_duration_s":2.513714,"scheduling_latency_s":2.513735,"redis_calls":13,"redis_duration_s":0.00282,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014778,"completed_at":"2026-07-09T06:51:26.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 159.778096] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:26.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c8c3af900516543bb7d70c6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c8c3af900516543bb7d70c6: start","job_status":"start","queue_duration_s":2.529759,"scheduling_latency_s":2.529779}1429gitlab # [ 160.069766] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1537438186 wd_nsec: 15374379581430gitlab # [ 159.988636] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c8c3af900516543bb7d70c6","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c8c3af900516543bb7d70c6: done: 2.748264 sec","job_status":"done","queue_duration_s":2.529759,"scheduling_latency_s":2.529779,"redis_calls":13,"redis_duration_s":2.72445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":2.721855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.748264,"completed_at":"2026-07-09T06:51:28.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 160.084759] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d85734382cce45a4f50c2c9a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d85734382cce45a4f50c2c9a: start","job_status":"start","queue_duration_s":5.280688,"scheduling_latency_s":5.28073}1432gitlab # [ 160.103943] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d85734382cce45a4f50c2c9a","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d85734382cce45a4f50c2c9a: done: 0.0208 sec","job_status":"done","queue_duration_s":5.280688,"scheduling_latency_s":5.28073,"redis_calls":13,"redis_duration_s":0.0048000000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003274,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0208,"completed_at":"2026-07-09T06:51:28.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 160.164424] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11d383d8ebb51b1a2953f370","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11d383d8ebb51b1a2953f370: start","job_status":"start","queue_duration_s":5.303163,"scheduling_latency_s":5.303183}1434gitlab # [ 160.165796] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11d383d8ebb51b1a2953f370","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11d383d8ebb51b1a2953f370: done: 0.01816 sec","job_status":"done","queue_duration_s":5.303163,"scheduling_latency_s":5.303183,"redis_calls":13,"redis_duration_s":0.005927,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002339,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003588,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01816,"completed_at":"2026-07-09T06:51:28.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 160.235537] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f5f5505d7e42100c134bd93","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f5f5505d7e42100c134bd93: start","job_status":"start","queue_duration_s":5.323253,"scheduling_latency_s":5.323274}1436gitlab # [ 160.236949] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f5f5505d7e42100c134bd93","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f5f5505d7e42100c134bd93: done: 0.018006 sec","job_status":"done","queue_duration_s":5.323253,"scheduling_latency_s":5.323274,"redis_calls":13,"redis_duration_s":0.005594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001688,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018006,"completed_at":"2026-07-09T06:51:28.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 160.254315] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f4a14946653c0ecc09995fd","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f4a14946653c0ecc09995fd: start","job_status":"start","queue_duration_s":5.343191,"scheduling_latency_s":5.343211}1438gitlab # [ 160.321754] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f4a14946653c0ecc09995fd","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f4a14946653c0ecc09995fd: done: 0.018022 sec","job_status":"done","queue_duration_s":5.343191,"scheduling_latency_s":5.343211,"redis_calls":13,"redis_duration_s":0.005545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018022,"completed_at":"2026-07-09T06:51:28.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 160.334158] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"102d700b81c42e0ab86da763","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-102d700b81c42e0ab86da763: start","job_status":"start","queue_duration_s":5.36301,"scheduling_latency_s":5.36303}1440gitlab # [ 160.418443] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"102d700b81c42e0ab86da763","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-102d700b81c42e0ab86da763: done: 0.018037 sec","job_status":"done","queue_duration_s":5.36301,"scheduling_latency_s":5.36303,"redis_calls":13,"redis_duration_s":0.005637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003801,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018037,"completed_at":"2026-07-09T06:51:28.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 160.420588] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bad3fca473a5cf0c950871d2","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bad3fca473a5cf0c950871d2: start","job_status":"start","queue_duration_s":5.382963,"scheduling_latency_s":5.382983}1442gitlab # [ 160.433549] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bad3fca473a5cf0c950871d2","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bad3fca473a5cf0c950871d2: done: 0.017771 sec","job_status":"done","queue_duration_s":5.382963,"scheduling_latency_s":5.382983,"redis_calls":13,"redis_duration_s":0.005244,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003503,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017771,"completed_at":"2026-07-09T06:51:28.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 160.514473] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bce18816cdf0e0be31290e3","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bce18816cdf0e0be31290e3: start","job_status":"start","queue_duration_s":5.402446,"scheduling_latency_s":5.402513}1444gitlab # [ 160.516171] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bce18816cdf0e0be31290e3","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bce18816cdf0e0be31290e3: done: 0.018373 sec","job_status":"done","queue_duration_s":5.402446,"scheduling_latency_s":5.402513,"redis_calls":13,"redis_duration_s":0.0057669999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018373,"completed_at":"2026-07-09T06:51:28.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 160.597120] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cdae44d0fc6f4f1a1883505","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cdae44d0fc6f4f1a1883505: start","job_status":"start","queue_duration_s":5.422853,"scheduling_latency_s":5.422874}1446gitlab # [ 160.598321] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cdae44d0fc6f4f1a1883505","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cdae44d0fc6f4f1a1883505: done: 0.027955 sec","job_status":"done","queue_duration_s":5.422853,"scheduling_latency_s":5.422874,"redis_calls":13,"redis_duration_s":0.0056630000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027955,"completed_at":"2026-07-09T06:51:28.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 160.667074] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"338889a05e8dc6cdd33f5269","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-338889a05e8dc6cdd33f5269: start","job_status":"start","queue_duration_s":5.453284,"scheduling_latency_s":5.453327}1448gitlab # [ 160.672311] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"338889a05e8dc6cdd33f5269","created_at":"2026-07-09T06:51:23.407Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-c53d81dabbe14850","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ea469cf2b789cdea25437f729e6afabe","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:23.513Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-338889a05e8dc6cdd33f5269: done: 0.020417 sec","job_status":"done","queue_duration_s":5.453284,"scheduling_latency_s":5.453327,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020417,"completed_at":"2026-07-09T06:51:28.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 160.743136] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:28.988Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b8de33e496fc322fc7c325d7","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b8de33e496fc322fc7c325d7: start","job_status":"start","queue_duration_s":4.040666,"scheduling_latency_s":4.040686}1450gitlab # [ 160.810269] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.003Z","message":"No existing merge request to be cleaned up."}1451gitlab # [ 160.810994] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.017Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b8de33e496fc322fc7c325d7","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b8de33e496fc322fc7c325d7: done: 0.028943 sec","job_status":"done","queue_duration_s":4.040666,"scheduling_latency_s":4.040686,"redis_calls":13,"redis_duration_s":0.003119,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001284,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001374,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000436089,"duration_s":0.028943,"completed_at":"2026-07-09T06:51:29.017Z","load_balancing_strategy":"primary","db_duration_s":0.005962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 160.814052] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.019Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38ccdc4a060495e49d56db03","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38ccdc4a060495e49d56db03: start","job_status":"start","queue_duration_s":4.071574,"scheduling_latency_s":4.071596}1453gitlab # [ 160.833207] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.030Z","message":"No existing merge request to be cleaned up."}1454gitlab # [ 160.836052] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.038Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38ccdc4a060495e49d56db03","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38ccdc4a060495e49d56db03: done: 0.019516 sec","job_status":"done","queue_duration_s":4.071574,"scheduling_latency_s":4.071596,"redis_calls":10,"redis_duration_s":0.004029,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001446,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002583,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019516,"completed_at":"2026-07-09T06:51:29.038Z","load_balancing_strategy":"primary","db_duration_s":0.002667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 160.919102] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.040Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8a565d7359f533f33d2e14d2","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8a565d7359f533f33d2e14d2: start","job_status":"start","queue_duration_s":4.092539,"scheduling_latency_s":4.092559}1456gitlab # [ 160.920383] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.049Z","message":"No existing merge request to be cleaned up."}1457gitlab # [ 160.921299] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.055Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8a565d7359f533f33d2e14d2","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8a565d7359f533f33d2e14d2: done: 0.015923 sec","job_status":"done","queue_duration_s":4.092539,"scheduling_latency_s":4.092559,"redis_calls":10,"redis_duration_s":0.0024720000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001544,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015923,"completed_at":"2026-07-09T06:51:29.055Z","load_balancing_strategy":"primary","db_duration_s":0.001954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 161.012117] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.057Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cdf6fcd7662562740a4f4044","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cdf6fcd7662562740a4f4044: start","job_status":"start","queue_duration_s":4.109756,"scheduling_latency_s":4.109776}1459gitlab # [ 161.013107] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.066Z","message":"No existing merge request to be cleaned up."}1460gitlab # [ 161.013982] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:51:29.072Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cdf6fcd7662562740a4f4044","created_at":"2026-07-09T06:51:24.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-874f2f490a5e4d96","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"b8c8cfdd998bae0dd8a07b99dcb16bc5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:51:24.947Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cdf6fcd7662562740a4f4044: done: 0.015352 sec","job_status":"done","queue_duration_s":4.109756,"scheduling_latency_s":4.109776,"redis_calls":10,"redis_duration_s":0.002524,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015352,"completed_at":"2026-07-09T06:51:29.072Z","load_balancing_strategy":"primary","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 171.812125] setup-gitlab-start[2748]: W, [2026-07-09T06:51:40.614493 #2748] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1462gitlab # [ 194.047240] runuser[2747]: pam_unix(runuser:session): session closed for user gitlab1463gitlab # [ 194.051842] setup-gitlab-start[2968]: +++ date -d '+7 days' +%F1464gitlab # [ 194.056101] setup-gitlab-start[2967]: ++ jq -r .token1465gitlab # [ 194.059537] setup-gitlab-start[2966]: ++ curl -fs -X POST -H 'Authorization: Bearer 1ada74272ecd2e25f1eb27e88b9ead10e8deb7835364224985b565ea2981f204' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-16"}' http://gitlab/api/v4/users/1/personal_access_tokens1466gitlab # [ 194.526216] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:03.328Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:03.321Z","job_size_bytes":451,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: start","job_status":"start","queue_duration_s":0.006428,"scheduling_latency_s":0.006492}1467gitlab # [ 194.537856] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:03 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 4691468gitlab # [ 194.592582] setup-gitlab-start[2706]: + TOKEN=glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj1469gitlab # [ 194.594391] setup-gitlab-start[2706]: + test -n glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj1470gitlab # [ 194.595960] setup-gitlab-start[2706]: + test glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj '!=' null1471gitlab # [ 194.597228] setup-gitlab-start[2706]: + mkdir -p /var/lib/secrets1472gitlab # [ 194.599104] setup-gitlab-start[2706]: + echo glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj1473gitlab # [ 194.600371] setup-gitlab-start[2706]: + chmod 644 /var/lib/secrets/gitlab-token1474gitlab # [ 194.602131] setup-gitlab-start[2706]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1475gitlab # [ 194.753776] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:03.556Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:03.321Z","job_size_bytes":451,"pid":2147,"sidekiq_tid":"lymr","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: fail: 0.227483 sec","job_status":"fail","queue_duration_s":0.006428,"scheduling_latency_s":0.006492,"redis_calls":11,"redis_duration_s":0.0059180000000000005,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001158,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000823,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00112193,"duration_s":0.227483,"completed_at":"2026-07-09T06:52:03.555Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.042849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 196.735332] gitaly[1397]: time="2026-07-09T06:52:05.537Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.537" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1477gitlab # [ 196.740198] gitaly[1397]: time="2026-07-09T06:52:05.538Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.537" grpc.time_ms=0.432 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1478gitlab # [ 196.924720] gitaly[1397]: time="2026-07-09T06:52:05.728Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.727" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1479gitlab # [ 196.929317] gitaly[1397]: time="2026-07-09T06:52:05.728Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.196" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.727" grpc.time_ms=0.61 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1480gitlab # [ 196.970421] gitaly[1397]: time="2026-07-09T06:52:05.773Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A diskcache=c5eb5297-68f8-4cac-90e4-516adf58383f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.210" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.741" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1481gitlab # [ 196.973638] gitaly[1397]: time="2026-07-09T06:52:05.774Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.210" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.741" grpc.time_ms=32.721 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1482gitlab # [ 196.988611] gitaly[1397]: time="2026-07-09T06:52:05.791Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.260" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.791" pid=1397 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1483gitlab # [ 196.989938] gitaly[1397]: time="2026-07-09T06:52:05.792Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.260" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.791" grpc.time_ms=0.602 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1484gitlab # [ 197.002164] gitaly[1397]: time="2026-07-09T06:52:05.805Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:15.267" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.797" grpc.time_ms=7.565 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1485gitlab # [ 197.076366] gitaly[1397]: time="2026-07-09T06:52:05.852Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.852" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1486gitlab # [ 197.077786] gitaly[1397]: time="2026-07-09T06:52:05.852Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.852" grpc.time_ms=0.29 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1487gitlab # [ 197.079119] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:05.836Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"31c67b912985432b0968368c","created_at":"2026-07-09T06:52:05.818Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.834Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-31c67b912985432b0968368c: start","job_status":"start","queue_duration_s":0.002522,"scheduling_latency_s":0.002578}1488gitlab # [ 197.117618] gitaly[1397]: time="2026-07-09T06:52:05.921Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A diskcache=383d173d-a884-474e-ab0c-e0833a9751b0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.855" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 197.118957] gitaly[1397]: time="2026-07-09T06:52:05.921Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=32 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-09T06:52:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.855" grpc.time_ms=65.859 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 197.126472] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:05.929Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"31c67b912985432b0968368c","created_at":"2026-07-09T06:52:05.818Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.834Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-31c67b912985432b0968368c: done: 0.09332 sec","job_status":"done","queue_duration_s":0.002522,"scheduling_latency_s":0.002578,"redis_calls":10,"redis_duration_s":0.003646,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001829,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.068455907,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.09332,"completed_at":"2026-07-09T06:52:05.929Z","load_balancing_strategy":"primary","db_duration_s":0.010906,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1491gitlab # [ 197.217091] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:05.931Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7b19655c50d8c035edb7a2ce","created_at":"2026-07-09T06:52:05.840Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.848Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7b19655c50d8c035edb7a2ce: start","job_status":"start","queue_duration_s":0.083179,"scheduling_latency_s":0.083201}1492gitlab # [ 197.218250] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.009Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7b19655c50d8c035edb7a2ce","created_at":"2026-07-09T06:52:05.840Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:05.848Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7b19655c50d8c035edb7a2ce: done: 0.078545 sec","job_status":"done","queue_duration_s":0.083179,"scheduling_latency_s":0.083201,"redis_calls":14,"redis_duration_s":0.004509,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000972,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002453,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055458166,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.078545,"completed_at":"2026-07-09T06:52:06.009Z","load_balancing_strategy":"primary","db_duration_s":0.017648,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1493gitlab # [ 197.293412] gitaly[1397]: time="2026-07-09T06:52:05.932Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.081" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.932" pid=1397 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1494gitlab # [ 197.294782] gitaly[1397]: time="2026-07-09T06:52:05.932Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.081" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.932" grpc.time_ms=0.433 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1495gitlab # [ 197.303722] gitaly[1397]: time="2026-07-09T06:52:05.955Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.940" grpc.time_ms=14.619 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1496gitlab # [ 197.325906] gitaly[1397]: time="2026-07-09T06:52:05.977Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:15.126" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.977" grpc.time_ms=0.21 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1497gitlab # [ 197.404080] gitaly[1397]: time="2026-07-09T06:52:05.984Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:15.133" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:05.984" grpc.time_ms=0.23 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1498gitlab # [ 197.405409] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.124Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"98847b990321840b0384c3f1","created_at":"2026-07-09T06:52:06.117Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.123Z","job_size_bytes":5,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-98847b990321840b0384c3f1: start","job_status":"start","queue_duration_s":0.001533,"scheduling_latency_s":0.00157}1499gitlab # [ 197.406200] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.198Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"98847b990321840b0384c3f1","created_at":"2026-07-09T06:52:06.117Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.123Z","job_size_bytes":5,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-98847b990321840b0384c3f1: done: 0.073765 sec","job_status":"done","queue_duration_s":0.001533,"scheduling_latency_s":0.00157,"redis_calls":12,"redis_duration_s":0.0029980000000000002,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000759,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001775,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000393905,"duration_s":0.073765,"completed_at":"2026-07-09T06:52:06.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010177,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1500gitlab # [ 197.477261] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.200Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"cca36d43cf838746f70b6a58","created_at":"2026-07-09T06:52:06.143Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.147Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-cca36d43cf838746f70b6a58: start","job_status":"start","queue_duration_s":0.052521,"scheduling_latency_s":0.052542}1501gitlab # [ 197.478801] gitaly[1397]: time="2026-07-09T06:52:06.171Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBMYGSGABBX2Z29X6AT4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:16.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:06.164" grpc.time_ms=6.171 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1502gitlab # [ 197.602564] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.405Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"cca36d43cf838746f70b6a58","created_at":"2026-07-09T06:52:06.143Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.147Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-cca36d43cf838746f70b6a58: done: 0.20575 sec","job_status":"done","queue_duration_s":0.052521,"scheduling_latency_s":0.052542,"redis_calls":11,"redis_duration_s":0.002745,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000699,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001558,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.178,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.178,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.160493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346971,"duration_s":0.20575,"completed_at":"2026-07-09T06:52:06.405Z","load_balancing_strategy":"primary","db_duration_s":0.043868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 197.604339] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.407Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1e89820572d8eaf163cf445e","created_at":"2026-07-09T06:52:06.149Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.153Z","job_size_bytes":89,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1e89820572d8eaf163cf445e: start","job_status":"start","queue_duration_s":0.253797,"scheduling_latency_s":0.253818}1504gitlab # [ 197.686819] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.454Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1e89820572d8eaf163cf445e","created_at":"2026-07-09T06:52:06.149Z","trace_propagation_headers":{"sentry-trace":"eb0872a5860b4966b86f2bd97e1acd50-d576c8b34cea4aeb","baggage":"sentry-trace_id=eb0872a5860b4966b86f2bd97e1acd50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMYGSGABBX2Z29X6AT4A","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.153Z","job_size_bytes":89,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1e89820572d8eaf163cf445e: done: 0.047462 sec","job_status":"done","queue_duration_s":0.253797,"scheduling_latency_s":0.253818,"redis_calls":11,"redis_duration_s":0.002551,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00066,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000447,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353118,"duration_s":0.047462,"completed_at":"2026-07-09T06:52:06.454Z","load_balancing_strategy":"primary","db_duration_s":0.014696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 197.707646] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:06 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 31011506gitlab # [ 197.709510] setup-gitlab-start[2706]: + mkdir -p /tmp/test-flake1507gitlab # [ 197.710852] setup-gitlab-start[2706]: + cd /tmp/test-flake1508gitlab # [ 197.711971] setup-gitlab-start[2706]: + git init -b master1509gitlab # [ 197.772233] setup-gitlab-start[3012]: Initialized empty Git repository in /tmp/test-flake/.git/1510gitlab # [ 197.773458] setup-gitlab-start[2706]: + git config user.name test1511gitlab # [ 197.777883] setup-gitlab-start[2706]: + git config user.email test@example.com1512gitlab # [ 197.781284] setup-gitlab-start[2706]: + cat1513gitlab # [ 197.783347] setup-gitlab-start[2706]: + git add flake.nix1514gitlab # [ 197.787326] setup-gitlab-start[2706]: + git commit -m 'initial commit'1515gitlab # [ 197.794353] setup-gitlab-start[3017]: [master (root-commit) 8cb3788] initial commit1516gitlab # [ 197.796126] setup-gitlab-start[3017]: 1 file changed, 10 insertions(+)1517gitlab # [ 197.797976] setup-gitlab-start[3017]: create mode 100644 flake.nix1518gitlab # [ 197.800236] setup-gitlab-start[2706]: + git remote add origin http://oauth2:glpat-nWLZ7BHMB06EMBQPc4P2Pm86MQp1OjEH.01.0w1usiroj@gitlab/root/test-flake.git1519gitlab # [ 197.804272] setup-gitlab-start[2706]: + git push -u origin master1520gitlab # [ 197.865217] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24e96ccdafcd80ec3b3d3341","created_at":"2026-07-09T06:52:06.661Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad9d3e47d7624121189f743f3a202b59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:52:06.665Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24e96ccdafcd80ec3b3d3341: start","job_status":"start","queue_duration_s":0.001823,"scheduling_latency_s":0.001919}1521gitlab # [ 197.923965] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24e96ccdafcd80ec3b3d3341","created_at":"2026-07-09T06:52:06.661Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad9d3e47d7624121189f743f3a202b59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-09T06:52:06.665Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24e96ccdafcd80ec3b3d3341: done: 0.059511 sec","job_status":"done","queue_duration_s":0.001823,"scheduling_latency_s":0.001919,"redis_calls":9,"redis_duration_s":0.009823,"redis_read_bytes":7,"redis_write_bytes":1018,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00103,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":277,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059511,"completed_at":"2026-07-09T06:52:06.727Z","load_balancing_strategy":"primary","db_duration_s":0.005337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 197.943615] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 1031523gitlab # [ 197.954100] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9147061b0f9adf512b97c1b7","created_at":"2026-07-09T06:52:06.696Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4504040db5ff04f5464f6426c388c90","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.703Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9147061b0f9adf512b97c1b7: start","job_status":"start","queue_duration_s":0.053241,"scheduling_latency_s":0.053275}1524gitlab # [ 197.974774] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.777Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1525gitlab # [ 198.013697] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9147061b0f9adf512b97c1b7","created_at":"2026-07-09T06:52:06.696Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4504040db5ff04f5464f6426c388c90","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.703Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9147061b0f9adf512b97c1b7: done: 0.058314 sec","job_status":"done","queue_duration_s":0.053241,"scheduling_latency_s":0.053275,"redis_calls":7,"redis_duration_s":0.014950000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014467,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058314,"completed_at":"2026-07-09T06:52:06.814Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 198.021128] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a30e018c14ef1fd4b4235201","created_at":"2026-07-09T06:52:06.718Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c36be163c6308060e191ff086c3561d6","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.722Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a30e018c14ef1fd4b4235201: start","job_status":"start","queue_duration_s":0.098891,"scheduling_latency_s":0.098914}1527gitlab # [ 198.060138] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a30e018c14ef1fd4b4235201","created_at":"2026-07-09T06:52:06.718Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c36be163c6308060e191ff086c3561d6","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.722Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a30e018c14ef1fd4b4235201: done: 0.041839 sec","job_status":"done","queue_duration_s":0.098891,"scheduling_latency_s":0.098914,"redis_calls":10,"redis_duration_s":0.007405,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002224,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005181,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015120916,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.041839,"completed_at":"2026-07-09T06:52:06.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014564,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1528gitlab # [ 198.158781] gitaly[1397]: time="2026-07-09T06:52:06.903Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBR6W49PB5QEQQHYW8E87 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:06.887" grpc.time_ms=15.636 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1529gitlab # [ 198.161415] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1561530gitlab # [ 198.163469] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"75a448ce9f4450ea35a658f3","created_at":"2026-07-09T06:52:06.753Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6b156631dc9e6c7de6cf829c670a11b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.760Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-75a448ce9f4450ea35a658f3: start","job_status":"start","queue_duration_s":0.103998,"scheduling_latency_s":0.104018}1531gitlab # [ 198.166157] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"75a448ce9f4450ea35a658f3","created_at":"2026-07-09T06:52:06.753Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6b156631dc9e6c7de6cf829c670a11b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.760Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-75a448ce9f4450ea35a658f3: done: 0.018766 sec","job_status":"done","queue_duration_s":0.103998,"scheduling_latency_s":0.104018,"redis_calls":12,"redis_duration_s":0.003625,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000876,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000636,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000387759,"duration_s":0.018766,"completed_at":"2026-07-09T06:52:06.883Z","load_balancing_strategy":"primary","db_duration_s":0.000541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 198.235146] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9d414adfc92895953dd42831","created_at":"2026-07-09T06:52:06.771Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cedb24bb71d4de55bd0f8403bd86204","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.783Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9d414adfc92895953dd42831: start","job_status":"start","queue_duration_s":0.10078,"scheduling_latency_s":0.100801}1533gitlab # [ 198.236397] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9d414adfc92895953dd42831","created_at":"2026-07-09T06:52:06.771Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cedb24bb71d4de55bd0f8403bd86204","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.783Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9d414adfc92895953dd42831: done: 0.018077 sec","job_status":"done","queue_duration_s":0.10078,"scheduling_latency_s":0.100801,"redis_calls":7,"redis_duration_s":0.00481,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018077,"completed_at":"2026-07-09T06:52:06.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 198.312315] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a96ba83ac23398c35597ba0f","created_at":"2026-07-09T06:52:06.798Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca07d2cc46704fac8b9096172ea0bd07","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.801Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a96ba83ac23398c35597ba0f: start","job_status":"start","queue_duration_s":0.103058,"scheduling_latency_s":0.103078}1535gitlab # [ 198.322645] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a96ba83ac23398c35597ba0f","created_at":"2026-07-09T06:52:06.798Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca07d2cc46704fac8b9096172ea0bd07","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.801Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a96ba83ac23398c35597ba0f: done: 0.048228 sec","job_status":"done","queue_duration_s":0.103058,"scheduling_latency_s":0.103078,"redis_calls":9,"redis_duration_s":0.003617,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000498388,"exclusive_lock_hold_duration_s":0.004753397,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.048228,"completed_at":"2026-07-09T06:52:06.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 198.435466] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:06.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cd4d1da12d4e6195d80ade08","created_at":"2026-07-09T06:52:06.812Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"336ee9b3f461fe1b45a493565957a5e3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.816Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cd4d1da12d4e6195d80ade08: start","job_status":"start","queue_duration_s":0.145634,"scheduling_latency_s":0.145656}1537gitlab # [ 198.436781] gitaly[1397]: time="2026-07-09T06:52:07.123Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:17.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:07.112" grpc.time_ms=11.023 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1538gitlab # [ 198.440822] gitaly[1397]: time="2026-07-09T06:52:07.162Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-09T06:52:37.053" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-09T06:52:07.153" grpc.time_ms=8.263 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1539gitlab # [ 198.449893] gitaly[1397]: time="2026-07-09T06:52:07.180Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV duration_ms=121ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/allowed"1540gitlab # [ 198.460104] gitaly[1397]: time="2026-07-09T06:52:07.262Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV duration_ms=81ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/pre_receive"1541gitlab # [ 198.460852] gitaly[1397]: time="2026-07-09T06:52:07.263Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.058" grpc.time_ms=204.574 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1542gitlab # [ 198.541926] gitaly[1397]: time="2026-07-09T06:52:07.292Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.292" grpc.time_ms=0.606 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1543gitlab # [ 198.546193] gitaly[1397]: time="2026-07-09T06:52:07.323Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.323" grpc.time_ms=0.455 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1544gitlab # [ 198.553680] gitaly[1397]: time="2026-07-09T06:52:07.356Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.352" grpc.time_ms=2.755 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1545gitlab # [ 198.650157] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cd4d1da12d4e6195d80ade08","created_at":"2026-07-09T06:52:06.812Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"336ee9b3f461fe1b45a493565957a5e3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.816Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cd4d1da12d4e6195d80ade08: done: 0.491018 sec","job_status":"done","queue_duration_s":0.145634,"scheduling_latency_s":0.145656,"redis_calls":571,"redis_duration_s":0.209126,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.207898,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.305676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.491018,"completed_at":"2026-07-09T06:52:07.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 198.657322] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"bd86919a48e36e2aa32394a6","created_at":"2026-07-09T06:52:06.828Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"006beaa50655fce307bc68cb594185ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.833Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-bd86919a48e36e2aa32394a6: start","job_status":"start","queue_duration_s":0.621287,"scheduling_latency_s":0.621315}1547gitlab # [ 198.673167] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"bd86919a48e36e2aa32394a6","created_at":"2026-07-09T06:52:06.828Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"006beaa50655fce307bc68cb594185ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.833Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-bd86919a48e36e2aa32394a6: done: 0.021822 sec","job_status":"done","queue_duration_s":0.621287,"scheduling_latency_s":0.621315,"redis_calls":9,"redis_duration_s":0.0051660000000000005,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000487771,"exclusive_lock_hold_duration_s":0.005938464,"duration_s":0.021822,"completed_at":"2026-07-09T06:52:07.476Z","load_balancing_strategy":"primary","db_duration_s":0.004193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548gitlab # [ 198.775885] gitaly[1397]: time="2026-07-09T06:52:07.498Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:17.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:07.477" grpc.time_ms=20.521 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1549gitlab # [ 198.777359] setup-gitlab-start[3021]: To http://gitlab/root/test-flake.git1550gitlab # [ 198.778176] setup-gitlab-start[3021]: * [new branch] master -> master1551gitlab # [ 198.778988] setup-gitlab-start[3021]: branch 'master' set up to track 'origin/master'.1552gitlab # [ 198.779865] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"48a35d3f5ae2742e1fb675c1","created_at":"2026-07-09T06:52:06.841Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41b76d1e42a2ff3729965f8b3a10acfe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:52:06.845Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-48a35d3f5ae2742e1fb675c1: start","job_status":"start","queue_duration_s":0.63573,"scheduling_latency_s":0.63576}1553gitlab # [ 198.780979] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.489Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1554gitlab # [ 198.787132] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"48a35d3f5ae2742e1fb675c1","created_at":"2026-07-09T06:52:06.841Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41b76d1e42a2ff3729965f8b3a10acfe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-09T06:52:06.845Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-48a35d3f5ae2742e1fb675c1: done: 0.014889 sec","job_status":"done","queue_duration_s":0.63573,"scheduling_latency_s":0.63576,"redis_calls":7,"redis_duration_s":0.003571,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014889,"completed_at":"2026-07-09T06:52:07.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 198.871145] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.498Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c60e2a1fad745992f745fa2a","created_at":"2026-07-09T06:52:06.851Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"cb68b841149f620c8a5520fe086b1dad","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.900Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c60e2a1fad745992f745fa2a: start","job_status":"start","queue_duration_s":0.59868,"scheduling_latency_s":0.598701}1556gitlab # [ 198.872389] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c60e2a1fad745992f745fa2a","created_at":"2026-07-09T06:52:06.851Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"cb68b841149f620c8a5520fe086b1dad","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.900Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c60e2a1fad745992f745fa2a: done: 0.020657 sec","job_status":"done","queue_duration_s":0.59868,"scheduling_latency_s":0.598701,"redis_calls":10,"redis_duration_s":0.002518,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000393067,"duration_s":0.020657,"completed_at":"2026-07-09T06:52:07.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 198.938896] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2c5e8926c0dfb0e0842572cd","created_at":"2026-07-09T06:52:06.915Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.918Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2c5e8926c0dfb0e0842572cd: start","job_status":"start","queue_duration_s":0.60455,"scheduling_latency_s":0.604579}1558gitlab # [ 199.015463] gitlab-workhorse[1306]: time="2026-07-09T06:52:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521559gitlab # [ 199.018578] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:07 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6601560gitlab: (finished: waiting for unit setup-gitlab.service, in 199.95 seconds)1561gitlab # [ 199.020138] gitaly[1397]: time="2026-07-09T06:52:07.541Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-09T06:52:17.049" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:07.519" grpc.time_ms=21.266 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1562gitlab: waiting for unit nixbot.service1563gitlab # [ 199.020934] gitaly[1397]: time="2026-07-09T06:52:07.560Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV duration_ms=173ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/post_receive"1564gitlab # [ 199.022460] systemd[1]: Finished setup-gitlab.service.1565gitlab # [ 199.023497] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2c5e8926c0dfb0e0842572cd","created_at":"2026-07-09T06:52:06.915Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.918Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2c5e8926c0dfb0e0842572cd: done: 0.081951 sec","job_status":"done","queue_duration_s":0.60455,"scheduling_latency_s":0.604579,"redis_calls":60,"redis_duration_s":0.016019000000000002,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000632,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.014394,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.081951,"completed_at":"2026-07-09T06:52:07.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566gitlab # [ 199.047854] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6f0a81b31ea5c2a95b06971f","created_at":"2026-07-09T06:52:06.922Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8b2b54ab96107ef21b0a651cb04e16b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.932Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6f0a81b31ea5c2a95b06971f: start","job_status":"start","queue_duration_s":0.674654,"scheduling_latency_s":0.674681}1567gitlab # [ 199.122413] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6f0a81b31ea5c2a95b06971f","created_at":"2026-07-09T06:52:06.922Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8b2b54ab96107ef21b0a651cb04e16b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.932Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6f0a81b31ea5c2a95b06971f: done: 0.190277 sec","job_status":"done","queue_duration_s":0.674654,"scheduling_latency_s":0.674681,"redis_calls":11,"redis_duration_s":0.002704,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000471009,"exclusive_lock_hold_duration_s":0.172752403,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.190277,"completed_at":"2026-07-09T06:52:07.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568gitlab # [ 199.133405] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d5ba35489ab709a532a46788","created_at":"2026-07-09T06:52:06.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e03ed705f4c4227e34e9e3996fd20c59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.941Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d5ba35489ab709a532a46788: start","job_status":"start","queue_duration_s":0.857317,"scheduling_latency_s":0.857338}1569gitlab # [ 199.200094] gitaly[1397]: time="2026-07-09T06:52:07.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:07.386" grpc.time_ms=174.807 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1570gitlab # [ 199.201325] gitaly[1397]: time="2026-07-09T06:52:07.572Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV diskcache=7c68ebf2-4f0d-4567-8733-75c11e4006fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:06.994" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1571gitlab # [ 199.202223] systemd[1]: Starting nixbot CI service...1572gitlab # [ 199.203419] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.829Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d5ba35489ab709a532a46788","created_at":"2026-07-09T06:52:06.936Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e03ed705f4c4227e34e9e3996fd20c59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.941Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d5ba35489ab709a532a46788: done: 0.030526 sec","job_status":"done","queue_duration_s":0.857317,"scheduling_latency_s":0.857338,"redis_calls":9,"redis_duration_s":0.002431,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445866,"exclusive_lock_hold_duration_s":0.016370517,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.030526,"completed_at":"2026-07-09T06:52:07.829Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 199.290173] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7bae42e74bf38cfe7a0b521d","created_at":"2026-07-09T06:52:06.953Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c07c5301136c2f7fbf48eae3e5c5d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.955Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7bae42e74bf38cfe7a0b521d: start","job_status":"start","queue_duration_s":0.874865,"scheduling_latency_s":0.874929}1574gitlab # [ 199.291972] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.839Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1575gitlab # [ 199.293050] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7bae42e74bf38cfe7a0b521d","created_at":"2026-07-09T06:52:06.953Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c07c5301136c2f7fbf48eae3e5c5d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.955Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7bae42e74bf38cfe7a0b521d: done: 0.012823 sec","job_status":"done","queue_duration_s":0.874865,"scheduling_latency_s":0.874929,"redis_calls":7,"redis_duration_s":0.0017140000000000002,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012823,"completed_at":"2026-07-09T06:52:07.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576gitlab # [ 199.365292] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b1b08eeb3d1a3b70da81fc29","created_at":"2026-07-09T06:52:06.964Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34a5125bdfe2d0db9f53574bd23894d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.965Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b1b08eeb3d1a3b70da81fc29: start","job_status":"start","queue_duration_s":0.878975,"scheduling_latency_s":0.878995}1577gitlab # [ 199.366854] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b1b08eeb3d1a3b70da81fc29","created_at":"2026-07-09T06:52:06.964Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34a5125bdfe2d0db9f53574bd23894d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.965Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b1b08eeb3d1a3b70da81fc29: done: 0.012189 sec","job_status":"done","queue_duration_s":0.878975,"scheduling_latency_s":0.878995,"redis_calls":7,"redis_duration_s":0.001605,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012189,"completed_at":"2026-07-09T06:52:07.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578gitlab # [ 199.435188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b8c4c54c5143f868a236ea2c","created_at":"2026-07-09T06:52:06.989Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d1cdff36c8527dc244a1fc292053a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.993Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b8c4c54c5143f868a236ea2c: start","job_status":"start","queue_duration_s":0.864423,"scheduling_latency_s":0.864444}1579gitlab # [ 199.510410] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.865Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1580gitlab # [ 199.511658] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b8c4c54c5143f868a236ea2c","created_at":"2026-07-09T06:52:06.989Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d1cdff36c8527dc244a1fc292053a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:06.993Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b8c4c54c5143f868a236ea2c: done: 0.012069 sec","job_status":"done","queue_duration_s":0.864423,"scheduling_latency_s":0.864444,"redis_calls":7,"redis_duration_s":0.00183,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000671,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012069,"completed_at":"2026-07-09T06:52:07.870Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581gitlab # [ 199.517063] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.871Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b6f1a9505451f5807fe61400","created_at":"2026-07-09T06:52:07.002Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3959cca50529f64ccc0b084acb05c6de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:52:07.006Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b6f1a9505451f5807fe61400: start","job_status":"start","queue_duration_s":0.864962,"scheduling_latency_s":0.864982}1582gitlab # [ 199.586418] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.879Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1583gitlab # [ 199.587884] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b6f1a9505451f5807fe61400","created_at":"2026-07-09T06:52:07.002Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3959cca50529f64ccc0b084acb05c6de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-09T06:52:07.006Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b6f1a9505451f5807fe61400: done: 0.011651 sec","job_status":"done","queue_duration_s":0.864962,"scheduling_latency_s":0.864982,"redis_calls":7,"redis_duration_s":0.00157,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011651,"completed_at":"2026-07-09T06:52:07.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584gitlab # [ 199.661298] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"37c3aa7d2250ada6677f2864","created_at":"2026-07-09T06:52:07.024Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b50b12011fae63fd3c85bfc874b7163c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.035Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-37c3aa7d2250ada6677f2864: start","job_status":"start","queue_duration_s":0.84885,"scheduling_latency_s":0.84887}1585gitlab # [ 199.662807] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"37c3aa7d2250ada6677f2864","created_at":"2026-07-09T06:52:07.024Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b50b12011fae63fd3c85bfc874b7163c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.035Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-37c3aa7d2250ada6677f2864: done: 0.020691 sec","job_status":"done","queue_duration_s":0.84885,"scheduling_latency_s":0.84887,"redis_calls":12,"redis_duration_s":0.002623,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000443352,"exclusive_lock_hold_duration_s":0.00421953,"duration_s":0.020691,"completed_at":"2026-07-09T06:52:07.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586gitlab # [ 199.734301] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c969a5d797899d7398f39983","created_at":"2026-07-09T06:52:07.049Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9967f72573b0284850b7450e4ea63d1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.051Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c969a5d797899d7398f39983: start","job_status":"start","queue_duration_s":0.854784,"scheduling_latency_s":0.854845}1587gitlab # [ 199.735796] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.923Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c969a5d797899d7398f39983","created_at":"2026-07-09T06:52:07.049Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9967f72573b0284850b7450e4ea63d1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.051Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c969a5d797899d7398f39983: done: 0.016936 sec","job_status":"done","queue_duration_s":0.854784,"scheduling_latency_s":0.854845,"redis_calls":7,"redis_duration_s":0.001701,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016936,"completed_at":"2026-07-09T06:52:07.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588gitlab # [ 199.803629] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a45375ff7c115532ebcc03a2","created_at":"2026-07-09T06:52:07.079Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.081Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a45375ff7c115532ebcc03a2: start","job_status":"start","queue_duration_s":0.843632,"scheduling_latency_s":0.843653}1589gitlab # [ 199.806578] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.944Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a45375ff7c115532ebcc03a2","created_at":"2026-07-09T06:52:07.079Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.081Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a45375ff7c115532ebcc03a2: done: 0.019808 sec","job_status":"done","queue_duration_s":0.843632,"scheduling_latency_s":0.843653,"redis_calls":14,"redis_duration_s":0.0031680000000000002,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000348,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002181,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019808,"completed_at":"2026-07-09T06:52:07.944Z","load_balancing_strategy":"primary","db_duration_s":0.000786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590gitlab # [ 199.888300] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0d4db9210771832a26d4bfd3","created_at":"2026-07-09T06:52:07.085Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf3cffa8950e35caba84b17f727d5b8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.089Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0d4db9210771832a26d4bfd3: start","job_status":"start","queue_duration_s":0.856584,"scheduling_latency_s":0.856605}1591gitlab # [ 199.889749] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0d4db9210771832a26d4bfd3","created_at":"2026-07-09T06:52:07.085Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf3cffa8950e35caba84b17f727d5b8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.089Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0d4db9210771832a26d4bfd3: done: 0.012903 sec","job_status":"done","queue_duration_s":0.856584,"scheduling_latency_s":0.856605,"redis_calls":7,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012903,"completed_at":"2026-07-09T06:52:07.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000774,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592gitlab # [ 199.974301] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ecbf54ad4f592214ae279d8f","created_at":"2026-07-09T06:52:07.103Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcab947599967a5aa99a0b65ea85ab1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.106Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ecbf54ad4f592214ae279d8f: start","job_status":"start","queue_duration_s":0.85362,"scheduling_latency_s":0.853641}1593gitlab # [ 199.975837] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.967Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1594gitlab # [ 199.976914] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ecbf54ad4f592214ae279d8f","created_at":"2026-07-09T06:52:07.103Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcab947599967a5aa99a0b65ea85ab1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.106Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ecbf54ad4f592214ae279d8f: done: 0.011425 sec","job_status":"done","queue_duration_s":0.85362,"scheduling_latency_s":0.853641,"redis_calls":7,"redis_duration_s":0.001548,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011425,"completed_at":"2026-07-09T06:52:07.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1595gitlab # [ 200.044307] gitaly[1397]: time="2026-07-09T06:52:07.573Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=141 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13100 command.oublock=32 command.real_time_ms=568 command.spawn_token_fork_ms=1 command.system_time_ms=87 command.user_time_ms=54 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:06.994" grpc.time_ms=578.412 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1596gitlab # [ 200.049424] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a07a7e493076332dca553af8","created_at":"2026-07-09T06:52:07.123Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2a8a018981418ee8313d0f375b84b55","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.131Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a07a7e493076332dca553af8: start","job_status":"start","queue_duration_s":0.84141,"scheduling_latency_s":0.841431}1597gitlab # [ 200.137953] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a07a7e493076332dca553af8","created_at":"2026-07-09T06:52:07.123Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2a8a018981418ee8313d0f375b84b55","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.131Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a07a7e493076332dca553af8: done: 0.024419 sec","job_status":"done","queue_duration_s":0.84141,"scheduling_latency_s":0.841431,"redis_calls":9,"redis_duration_s":0.002016,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000516546,"exclusive_lock_hold_duration_s":0.007834515,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.024419,"completed_at":"2026-07-09T06:52:07.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598gitlab # [ 200.142194] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:07.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0fae6701baf2f53220866eb7","created_at":"2026-07-09T06:52:07.164Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e47fe783120b551b611246f21acb64d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.167Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0fae6701baf2f53220866eb7: start","job_status":"start","queue_duration_s":0.830754,"scheduling_latency_s":0.830774}1599gitlab # [ 200.213221] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0fae6701baf2f53220866eb7","created_at":"2026-07-09T06:52:07.164Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e47fe783120b551b611246f21acb64d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.167Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0fae6701baf2f53220866eb7: done: 0.016134 sec","job_status":"done","queue_duration_s":0.830754,"scheduling_latency_s":0.830774,"redis_calls":7,"redis_duration_s":0.001771,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.016134,"completed_at":"2026-07-09T06:52:08.014Z","load_balancing_strategy":"primary","db_duration_s":0.00404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600gitlab # [ 200.285836] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"7136ba77f6fc3dccf678b89d","created_at":"2026-07-09T06:52:07.174Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ca8838d7194ae78aaa4879b40252bb3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.181Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-7136ba77f6fc3dccf678b89d: start","job_status":"start","queue_duration_s":0.834903,"scheduling_latency_s":0.834923}1601gitlab # [ 200.287108] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"7136ba77f6fc3dccf678b89d","created_at":"2026-07-09T06:52:07.174Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-58ad9c0cabab4597","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ca8838d7194ae78aaa4879b40252bb3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.181Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-7136ba77f6fc3dccf678b89d: done: 0.020348 sec","job_status":"done","queue_duration_s":0.834903,"scheduling_latency_s":0.834923,"redis_calls":10,"redis_duration_s":0.002136,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337194,"duration_s":0.020348,"completed_at":"2026-07-09T06:52:08.036Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602gitlab # [ 200.309496] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.037Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8cb378872b8fe1f7cde149c561f5802ee16ad182 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e86a6a54322f311f46ee7687","created_at":"2026-07-09T06:52:07.421Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-f59c66934d58426d","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.443Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e86a6a54322f311f46ee7687: start","job_status":"start","queue_duration_s":0.59383,"scheduling_latency_s":0.59385}1603gitlab # [ 200.382511] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.056Z","message":"changes: \"0000000000000000000000000000000000000000 8cb378872b8fe1f7cde149c561f5802ee16ad182 refs/heads/master\\n\""}1604gitlab # [ 200.383994] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.721Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8cb378872b8fe1f7cde149c561f5802ee16ad182 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e86a6a54322f311f46ee7687","created_at":"2026-07-09T06:52:07.421Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-f59c66934d58426d","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.443Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e86a6a54322f311f46ee7687: done: 0.683293 sec","job_status":"done","queue_duration_s":0.59383,"scheduling_latency_s":0.59385,"gitaly_calls":13,"gitaly_duration_s":0.161524,"redis_calls":201,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.043203000000000005,"redis_read_bytes":963,"redis_write_bytes":20009,"redis_cache_calls":14,"redis_cache_duration_s":0.00357,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":34,"redis_feature_flag_duration_s":0.007701,"redis_feature_flag_read_bytes":542,"redis_feature_flag_write_bytes":4562,"redis_queues_calls":8,"redis_queues_duration_s":0.001118,"redis_queues_read_bytes":12,"redis_queues_write_bytes":4932,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00201,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018138,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010666,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":53,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":53,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":53,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.25,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.267,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.432393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000498108,"exclusive_lock_hold_duration_s":0.0330285,"duration_s":0.683293,"completed_at":"2026-07-09T06:52:08.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068602,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1605gitlab # [ 200.463108] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf8e37c735507af83c31d11","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf8e37c735507af83c31d11: start","job_status":"start","queue_duration_s":1.123703,"scheduling_latency_s":1.123726}1606gitlab # [ 200.464295] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf8e37c735507af83c31d11","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf8e37c735507af83c31d11: done: 0.029111 sec","job_status":"done","queue_duration_s":1.123703,"scheduling_latency_s":1.123726,"redis_calls":26,"redis_duration_s":0.005818,"redis_read_bytes":150,"redis_write_bytes":2841,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002882,"redis_feature_flag_read_bytes":140,"redis_feature_flag_write_bytes":1684,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029111,"completed_at":"2026-07-09T06:52:08.751Z","load_balancing_strategy":"primary","db_duration_s":0.001915,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607gitlab # [ 200.499423] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38b0612f1b1c3eae6914c356","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38b0612f1b1c3eae6914c356: start","job_status":"start","queue_duration_s":1.154116,"scheduling_latency_s":1.154137}1608gitlab # [ 200.582376] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38b0612f1b1c3eae6914c356","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38b0612f1b1c3eae6914c356: done: 0.015283 sec","job_status":"done","queue_duration_s":1.154116,"scheduling_latency_s":1.154137,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015283,"completed_at":"2026-07-09T06:52:08.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1609gitlab # [ 200.584164] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0220c5ff88f5ba5aa3d459dd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0220c5ff88f5ba5aa3d459dd: start","job_status":"start","queue_duration_s":1.170887,"scheduling_latency_s":1.170907}1610gitlab # [ 200.598138] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0220c5ff88f5ba5aa3d459dd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0220c5ff88f5ba5aa3d459dd: done: 0.015573 sec","job_status":"done","queue_duration_s":1.170887,"scheduling_latency_s":1.170907,"redis_calls":13,"redis_duration_s":0.002873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015573,"completed_at":"2026-07-09T06:52:08.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1611gitlab # [ 200.677131] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7541d34c337bb99737bba584","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7541d34c337bb99737bba584: start","job_status":"start","queue_duration_s":1.18793,"scheduling_latency_s":1.187951}1612gitlab # [ 200.678438] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7541d34c337bb99737bba584","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7541d34c337bb99737bba584: done: 0.01556 sec","job_status":"done","queue_duration_s":1.18793,"scheduling_latency_s":1.187951,"redis_calls":13,"redis_duration_s":0.002731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01556,"completed_at":"2026-07-09T06:52:08.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613gitlab # [ 200.747214] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54a6f79d20287855d0a4edf0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54a6f79d20287855d0a4edf0: start","job_status":"start","queue_duration_s":1.205294,"scheduling_latency_s":1.205314}1614gitlab # [ 200.756045] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54a6f79d20287855d0a4edf0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54a6f79d20287855d0a4edf0: done: 0.015179 sec","job_status":"done","queue_duration_s":1.205294,"scheduling_latency_s":1.205314,"redis_calls":13,"redis_duration_s":0.002817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015179,"completed_at":"2026-07-09T06:52:08.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615gitlab # [ 200.826302] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728f56c6aea63a6e4d29afb7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728f56c6aea63a6e4d29afb7: start","job_status":"start","queue_duration_s":1.221708,"scheduling_latency_s":1.221729}1616gitlab # [ 200.834796] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728f56c6aea63a6e4d29afb7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728f56c6aea63a6e4d29afb7: done: 0.015049 sec","job_status":"done","queue_duration_s":1.221708,"scheduling_latency_s":1.221729,"redis_calls":13,"redis_duration_s":0.003183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015049,"completed_at":"2026-07-09T06:52:08.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1617gitlab # [ 200.915775] gitaly[1397]: time="2026-07-09T06:52:08.097Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:18.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.086" grpc.time_ms=10.271 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618gitlab # [ 200.916906] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c217cd1faa8a6db01a5961eb","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c217cd1faa8a6db01a5961eb: start","job_status":"start","queue_duration_s":1.238017,"scheduling_latency_s":1.238037}1619gitlab # [ 200.994942] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c217cd1faa8a6db01a5961eb","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c217cd1faa8a6db01a5961eb: done: 0.015182 sec","job_status":"done","queue_duration_s":1.238017,"scheduling_latency_s":1.238037,"redis_calls":13,"redis_duration_s":0.002886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015182,"completed_at":"2026-07-09T06:52:08.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620gitlab # [ 200.996308] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a22bcf7bc1d2e7aa60cf51","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a22bcf7bc1d2e7aa60cf51: start","job_status":"start","queue_duration_s":1.25446,"scheduling_latency_s":1.254491}1621gitlab # [ 201.062824] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89a22bcf7bc1d2e7aa60cf51","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89a22bcf7bc1d2e7aa60cf51: done: 0.015291 sec","job_status":"done","queue_duration_s":1.25446,"scheduling_latency_s":1.254491,"redis_calls":13,"redis_duration_s":0.002862,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015291,"completed_at":"2026-07-09T06:52:08.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622gitlab # [ 201.130435] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd30fc4d1ef5923e07020ccc","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd30fc4d1ef5923e07020ccc: start","job_status":"start","queue_duration_s":1.271454,"scheduling_latency_s":1.271475}1623gitlab # [ 201.131692] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd30fc4d1ef5923e07020ccc","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd30fc4d1ef5923e07020ccc: done: 0.016542 sec","job_status":"done","queue_duration_s":1.271454,"scheduling_latency_s":1.271475,"redis_calls":13,"redis_duration_s":0.003261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016542,"completed_at":"2026-07-09T06:52:08.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624gitlab # [ 201.199510] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d72352fa1815e50fc32ee36","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d72352fa1815e50fc32ee36: start","job_status":"start","queue_duration_s":1.289282,"scheduling_latency_s":1.289303}1625gitlab # [ 201.201049] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d72352fa1815e50fc32ee36","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d72352fa1815e50fc32ee36: done: 0.015314 sec","job_status":"done","queue_duration_s":1.289282,"scheduling_latency_s":1.289303,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015314,"completed_at":"2026-07-09T06:52:08.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626gitlab # [ 201.285840] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"975a0e9baf3061ed78282f1e","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-975a0e9baf3061ed78282f1e: start","job_status":"start","queue_duration_s":1.305883,"scheduling_latency_s":1.305904}1627gitlab # [ 201.286912] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"975a0e9baf3061ed78282f1e","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-975a0e9baf3061ed78282f1e: done: 0.01492 sec","job_status":"done","queue_duration_s":1.305883,"scheduling_latency_s":1.305904,"redis_calls":13,"redis_duration_s":0.0027949999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01492,"completed_at":"2026-07-09T06:52:08.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628gitlab # [ 201.363253] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75021a0320670f9ec3150cf7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75021a0320670f9ec3150cf7: start","job_status":"start","queue_duration_s":1.323471,"scheduling_latency_s":1.323492}1629gitlab # [ 201.364761] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75021a0320670f9ec3150cf7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75021a0320670f9ec3150cf7: done: 0.021487 sec","job_status":"done","queue_duration_s":1.323471,"scheduling_latency_s":1.323492,"redis_calls":13,"redis_duration_s":0.0035180000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021487,"completed_at":"2026-07-09T06:52:08.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630gitlab # [ 201.438315] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beca159aeba03a14c86d9227","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beca159aeba03a14c86d9227: start","job_status":"start","queue_duration_s":1.346304,"scheduling_latency_s":1.346324}1631gitlab # [ 201.439853] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beca159aeba03a14c86d9227","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beca159aeba03a14c86d9227: done: 0.018149 sec","job_status":"done","queue_duration_s":1.346304,"scheduling_latency_s":1.346324,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018149,"completed_at":"2026-07-09T06:52:08.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632gitlab # [ 201.506923] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cdde8edaf953770ab2859e5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cdde8edaf953770ab2859e5: start","job_status":"start","queue_duration_s":1.365985,"scheduling_latency_s":1.366011}1633gitlab # [ 201.508182] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cdde8edaf953770ab2859e5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cdde8edaf953770ab2859e5: done: 0.016289 sec","job_status":"done","queue_duration_s":1.365985,"scheduling_latency_s":1.366011,"redis_calls":13,"redis_duration_s":0.003035,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016289,"completed_at":"2026-07-09T06:52:08.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634gitlab # [ 201.577067] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98804ba2eeacb1d5ac2bf318","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98804ba2eeacb1d5ac2bf318: start","job_status":"start","queue_duration_s":1.384045,"scheduling_latency_s":1.384066}1635gitlab # [ 201.578289] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98804ba2eeacb1d5ac2bf318","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98804ba2eeacb1d5ac2bf318: done: 0.014919 sec","job_status":"done","queue_duration_s":1.384045,"scheduling_latency_s":1.384066,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-07-09T06:52:08.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636gitlab # [ 201.654850] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:08.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80ab7b97a1a08d6dabb0fe9","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80ab7b97a1a08d6dabb0fe9: start","job_status":"start","queue_duration_s":1.400175,"scheduling_latency_s":1.400194}1637gitlab # [ 201.656188] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80ab7b97a1a08d6dabb0fe9","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80ab7b97a1a08d6dabb0fe9: done: 0.014894 sec","job_status":"done","queue_duration_s":1.400175,"scheduling_latency_s":1.400194,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014894,"completed_at":"2026-07-09T06:52:09.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638gitlab # [ 201.730112] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9087bd94653eebe1d391552","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9087bd94653eebe1d391552: start","job_status":"start","queue_duration_s":1.416283,"scheduling_latency_s":1.416303}1639gitlab # [ 201.731660] gitaly[1397]: time="2026-07-09T06:52:08.107Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:18.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.101" grpc.time_ms=5.831 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1640gitlab # [ 201.799989] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9087bd94653eebe1d391552","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9087bd94653eebe1d391552: done: 0.018312 sec","job_status":"done","queue_duration_s":1.416283,"scheduling_latency_s":1.416303,"redis_calls":13,"redis_duration_s":0.003098,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018312,"completed_at":"2026-07-09T06:52:09.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1641gitlab # [ 201.801697] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"933093bb279fb6387de04b88","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-933093bb279fb6387de04b88: start","job_status":"start","queue_duration_s":1.43652,"scheduling_latency_s":1.436547}1642gitlab # [ 201.875123] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"933093bb279fb6387de04b88","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-933093bb279fb6387de04b88: done: 0.016051 sec","job_status":"done","queue_duration_s":1.43652,"scheduling_latency_s":1.436547,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016051,"completed_at":"2026-07-09T06:52:09.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1643gitlab # [ 201.877040] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754c0d491990cdf4f6ca63c8","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754c0d491990cdf4f6ca63c8: start","job_status":"start","queue_duration_s":1.453865,"scheduling_latency_s":1.453885}1644gitlab # [ 201.942928] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754c0d491990cdf4f6ca63c8","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754c0d491990cdf4f6ca63c8: done: 0.014912 sec","job_status":"done","queue_duration_s":1.453865,"scheduling_latency_s":1.453885,"redis_calls":13,"redis_duration_s":0.002758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014912,"completed_at":"2026-07-09T06:52:09.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1645gitlab # [ 202.015286] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ddb080480db1a8837c0eb89","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ddb080480db1a8837c0eb89: start","job_status":"start","queue_duration_s":1.470038,"scheduling_latency_s":1.470059}1646gitlab # [ 202.016338] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ddb080480db1a8837c0eb89","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ddb080480db1a8837c0eb89: done: 0.015027 sec","job_status":"done","queue_duration_s":1.470038,"scheduling_latency_s":1.470059,"redis_calls":13,"redis_duration_s":0.0028580000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015027,"completed_at":"2026-07-09T06:52:09.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1647gitlab # [ 202.093228] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b03437371298ef8f268ea35f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b03437371298ef8f268ea35f: start","job_status":"start","queue_duration_s":1.486492,"scheduling_latency_s":1.486512}1648gitlab # [ 202.094236] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b03437371298ef8f268ea35f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b03437371298ef8f268ea35f: done: 0.014719 sec","job_status":"done","queue_duration_s":1.486492,"scheduling_latency_s":1.486512,"redis_calls":13,"redis_duration_s":0.002777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014719,"completed_at":"2026-07-09T06:52:09.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1649gitlab # [ 202.167192] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6c795629e988f66dfbedb30","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6c795629e988f66dfbedb30: start","job_status":"start","queue_duration_s":1.502444,"scheduling_latency_s":1.502463}1650gitlab # [ 202.168411] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6c795629e988f66dfbedb30","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6c795629e988f66dfbedb30: done: 0.015045 sec","job_status":"done","queue_duration_s":1.502444,"scheduling_latency_s":1.502463,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015045,"completed_at":"2026-07-09T06:52:09.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1651gitlab # [ 202.187088] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"459ae322a275d66fb7589019","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-459ae322a275d66fb7589019: start","job_status":"start","queue_duration_s":1.51875,"scheduling_latency_s":1.518817}1652gitlab # [ 202.255181] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"459ae322a275d66fb7589019","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-459ae322a275d66fb7589019: done: 0.015422 sec","job_status":"done","queue_duration_s":1.51875,"scheduling_latency_s":1.518817,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015422,"completed_at":"2026-07-09T06:52:09.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1653gitlab # [ 202.265863] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"445887ed948df69f63b03f60","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-445887ed948df69f63b03f60: start","job_status":"start","queue_duration_s":1.535449,"scheduling_latency_s":1.535469}1654gitlab # [ 202.340549] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"445887ed948df69f63b03f60","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-445887ed948df69f63b03f60: done: 0.01505 sec","job_status":"done","queue_duration_s":1.535449,"scheduling_latency_s":1.535469,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01505,"completed_at":"2026-07-09T06:52:09.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1655gitlab # [ 202.343972] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e5372203dfda149432201f0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e5372203dfda149432201f0: start","job_status":"start","queue_duration_s":1.551932,"scheduling_latency_s":1.551952}1656gitlab # [ 202.424289] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e5372203dfda149432201f0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e5372203dfda149432201f0: done: 0.014761 sec","job_status":"done","queue_duration_s":1.551932,"scheduling_latency_s":1.551952,"redis_calls":13,"redis_duration_s":0.002731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014761,"completed_at":"2026-07-09T06:52:09.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1657gitlab # [ 202.426196] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e54c748368607d92157ecd61","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e54c748368607d92157ecd61: start","job_status":"start","queue_duration_s":1.567971,"scheduling_latency_s":1.56799}1658gitlab # [ 202.441888] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e54c748368607d92157ecd61","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e54c748368607d92157ecd61: done: 0.015021 sec","job_status":"done","queue_duration_s":1.567971,"scheduling_latency_s":1.56799,"redis_calls":13,"redis_duration_s":0.003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015021,"completed_at":"2026-07-09T06:52:09.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659gitlab # [ 202.513236] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0251998987fa90dc41f6064f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0251998987fa90dc41f6064f: start","job_status":"start","queue_duration_s":1.584215,"scheduling_latency_s":1.584235}1660gitlab # [ 202.522801] gitaly[1397]: time="2026-07-09T06:52:08.139Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-09T06:52:18.050" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.130" grpc.time_ms=7.98 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1661gitlab # [ 202.592118] gitaly[1397]: time="2026-07-09T06:52:08.173Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:08.173" grpc.time_ms=0.497 pid=1397 span.kind=server system=grpc1662gitlab # [ 202.593479] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0251998987fa90dc41f6064f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0251998987fa90dc41f6064f: done: 0.015156 sec","job_status":"done","queue_duration_s":1.584215,"scheduling_latency_s":1.584235,"redis_calls":13,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015156,"completed_at":"2026-07-09T06:52:09.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663gitlab # [ 202.677081] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"833757e6c8595916c8ef43ca","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-833757e6c8595916c8ef43ca: start","job_status":"start","queue_duration_s":1.60063,"scheduling_latency_s":1.600651}1664gitlab # [ 202.678312] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"833757e6c8595916c8ef43ca","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-833757e6c8595916c8ef43ca: done: 0.01512 sec","job_status":"done","queue_duration_s":1.60063,"scheduling_latency_s":1.600651,"redis_calls":13,"redis_duration_s":0.002817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01512,"completed_at":"2026-07-09T06:52:09.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1665gitlab # [ 202.747110] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e16af30851382b81bb139eb3","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e16af30851382b81bb139eb3: start","job_status":"start","queue_duration_s":1.617202,"scheduling_latency_s":1.617221}1666gitlab # [ 202.748334] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e16af30851382b81bb139eb3","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e16af30851382b81bb139eb3: done: 0.014964 sec","job_status":"done","queue_duration_s":1.617202,"scheduling_latency_s":1.617221,"redis_calls":13,"redis_duration_s":0.002805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014964,"completed_at":"2026-07-09T06:52:09.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1667gitlab # [ 202.819085] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0280c66caf57a8aee69e29b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0280c66caf57a8aee69e29b4: start","job_status":"start","queue_duration_s":1.634216,"scheduling_latency_s":1.634255}1668gitlab # [ 202.820102] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0280c66caf57a8aee69e29b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0280c66caf57a8aee69e29b4: done: 0.017811 sec","job_status":"done","queue_duration_s":1.634216,"scheduling_latency_s":1.634255,"redis_calls":13,"redis_duration_s":0.002837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017811,"completed_at":"2026-07-09T06:52:09.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1669gitlab # [ 202.891651] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0f138792b351212a7f8a69","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0f138792b351212a7f8a69: start","job_status":"start","queue_duration_s":1.65346,"scheduling_latency_s":1.65348}1670gitlab # [ 202.892694] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0f138792b351212a7f8a69","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0f138792b351212a7f8a69: done: 0.014886 sec","job_status":"done","queue_duration_s":1.65346,"scheduling_latency_s":1.65348,"redis_calls":13,"redis_duration_s":0.002697,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014886,"completed_at":"2026-07-09T06:52:09.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1671gitlab # [ 202.960871] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6525d7104232f77532f2b9d5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6525d7104232f77532f2b9d5: start","job_status":"start","queue_duration_s":1.669587,"scheduling_latency_s":1.669607}1672gitlab # [ 202.962217] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6525d7104232f77532f2b9d5","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6525d7104232f77532f2b9d5: done: 0.014919 sec","job_status":"done","queue_duration_s":1.669587,"scheduling_latency_s":1.669607,"redis_calls":13,"redis_duration_s":0.002706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-07-09T06:52:09.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1673gitlab # [ 203.032947] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc7eadc02550020c274e1b46","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc7eadc02550020c274e1b46: start","job_status":"start","queue_duration_s":1.68576,"scheduling_latency_s":1.685816}1674gitlab # [ 203.033909] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc7eadc02550020c274e1b46","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc7eadc02550020c274e1b46: done: 0.015045 sec","job_status":"done","queue_duration_s":1.68576,"scheduling_latency_s":1.685816,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015045,"completed_at":"2026-07-09T06:52:09.299Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1675gitlab # [ 203.102893] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c94b4f6156fe95a7d315888","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c94b4f6156fe95a7d315888: start","job_status":"start","queue_duration_s":1.702237,"scheduling_latency_s":1.702257}1676gitlab # [ 203.105096] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c94b4f6156fe95a7d315888","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c94b4f6156fe95a7d315888: done: 0.014877 sec","job_status":"done","queue_duration_s":1.702237,"scheduling_latency_s":1.702257,"redis_calls":13,"redis_duration_s":0.00279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014877,"completed_at":"2026-07-09T06:52:09.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1677gitlab # [ 203.175743] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af92daccbe040ef4d61efc94","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af92daccbe040ef4d61efc94: start","job_status":"start","queue_duration_s":1.718352,"scheduling_latency_s":1.718373}1678gitlab # [ 203.184760] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af92daccbe040ef4d61efc94","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af92daccbe040ef4d61efc94: done: 0.014555 sec","job_status":"done","queue_duration_s":1.718352,"scheduling_latency_s":1.718373,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014555,"completed_at":"2026-07-09T06:52:09.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1679gitlab # [ 203.290140] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ce4a9503797e1e2cb45c15","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ce4a9503797e1e2cb45c15: start","job_status":"start","queue_duration_s":1.734146,"scheduling_latency_s":1.734165}1680gitlab # [ 203.291381] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ce4a9503797e1e2cb45c15","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ce4a9503797e1e2cb45c15: done: 0.01464 sec","job_status":"done","queue_duration_s":1.734146,"scheduling_latency_s":1.734165,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01464,"completed_at":"2026-07-09T06:52:09.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1681gitlab # [ 203.365534] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0ff38f527d47d2867040110","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0ff38f527d47d2867040110: start","job_status":"start","queue_duration_s":1.750061,"scheduling_latency_s":1.750082}1682gitlab # [ 203.366831] gitaly[1397]: time="2026-07-09T06:52:08.196Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:08.196" grpc.time_ms=0.306 pid=1397 span.kind=server system=grpc1683gitlab # [ 203.368081] gitaly[1397]: time="2026-07-09T06:52:08.206Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV diskcache=0ea82ed2-32b4-44f2-bbdb-fd0e5e0ab01b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:18.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.148" pid=1397 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1684gitlab # [ 203.373179] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0ff38f527d47d2867040110","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0ff38f527d47d2867040110: done: 0.014623 sec","job_status":"done","queue_duration_s":1.750061,"scheduling_latency_s":1.750082,"redis_calls":13,"redis_duration_s":0.002802,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014623,"completed_at":"2026-07-09T06:52:09.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1685gitlab # [ 203.443157] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28a4e48a249b187f2d7f2de0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28a4e48a249b187f2d7f2de0: start","job_status":"start","queue_duration_s":1.766003,"scheduling_latency_s":1.766023}1686gitlab # [ 203.512149] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28a4e48a249b187f2d7f2de0","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28a4e48a249b187f2d7f2de0: done: 0.015036 sec","job_status":"done","queue_duration_s":1.766003,"scheduling_latency_s":1.766023,"redis_calls":13,"redis_duration_s":0.003006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015036,"completed_at":"2026-07-09T06:52:09.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1687gitlab # [ 203.513640] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6d26199f94105986664017b","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6d26199f94105986664017b: start","job_status":"start","queue_duration_s":1.782288,"scheduling_latency_s":1.782307}1688gitlab # [ 203.530054] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6d26199f94105986664017b","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6d26199f94105986664017b: done: 0.018477 sec","job_status":"done","queue_duration_s":1.782288,"scheduling_latency_s":1.782307,"redis_calls":13,"redis_duration_s":0.0030830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018477,"completed_at":"2026-07-09T06:52:09.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1689gitlab # [ 203.597946] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85fec29dd4cc6607ccfa439d","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85fec29dd4cc6607ccfa439d: start","job_status":"start","queue_duration_s":1.802266,"scheduling_latency_s":1.802288}1690gitlab # [ 203.608417] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85fec29dd4cc6607ccfa439d","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85fec29dd4cc6607ccfa439d: done: 0.015156 sec","job_status":"done","queue_duration_s":1.802266,"scheduling_latency_s":1.802288,"redis_calls":13,"redis_duration_s":0.0027229999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015156,"completed_at":"2026-07-09T06:52:09.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1691gitlab # [ 203.679894] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf0f070d5311e4d7b039e02","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf0f070d5311e4d7b039e02: start","job_status":"start","queue_duration_s":1.818632,"scheduling_latency_s":1.818652}1692gitlab # [ 203.751161] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf0f070d5311e4d7b039e02","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf0f070d5311e4d7b039e02: done: 0.01469 sec","job_status":"done","queue_duration_s":1.818632,"scheduling_latency_s":1.818652,"redis_calls":13,"redis_duration_s":0.002774,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01469,"completed_at":"2026-07-09T06:52:09.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1693gitlab # [ 203.752896] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92a01a4da994b99da4521ccf","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92a01a4da994b99da4521ccf: start","job_status":"start","queue_duration_s":1.834626,"scheduling_latency_s":1.834646}1694gitlab # [ 203.823097] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92a01a4da994b99da4521ccf","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92a01a4da994b99da4521ccf: done: 0.014967 sec","job_status":"done","queue_duration_s":1.834626,"scheduling_latency_s":1.834646,"redis_calls":13,"redis_duration_s":0.002859,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014967,"completed_at":"2026-07-09T06:52:09.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1695gitlab # [ 203.825203] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"305da8dd73a8ca051b9a0675","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-305da8dd73a8ca051b9a0675: start","job_status":"start","queue_duration_s":1.850942,"scheduling_latency_s":1.850962}1696gitlab # [ 203.893154] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"305da8dd73a8ca051b9a0675","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-305da8dd73a8ca051b9a0675: done: 0.014927 sec","job_status":"done","queue_duration_s":1.850942,"scheduling_latency_s":1.850962,"redis_calls":13,"redis_duration_s":0.0027359999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014927,"completed_at":"2026-07-09T06:52:09.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1697gitlab # [ 203.902987] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc60bb96f999377c60434b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc60bb96f999377c60434b4: start","job_status":"start","queue_duration_s":1.867131,"scheduling_latency_s":1.86715}1698gitlab # [ 203.975309] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc60bb96f999377c60434b4","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc60bb96f999377c60434b4: done: 0.015165 sec","job_status":"done","queue_duration_s":1.867131,"scheduling_latency_s":1.86715,"redis_calls":13,"redis_duration_s":0.0029899999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015165,"completed_at":"2026-07-09T06:52:09.481Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1699gitlab # [ 203.981858] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ce0105a1b91f2d402b7bda7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ce0105a1b91f2d402b7bda7: start","job_status":"start","queue_duration_s":1.883544,"scheduling_latency_s":1.883565}1700gitlab # [ 204.000505] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ce0105a1b91f2d402b7bda7","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ce0105a1b91f2d402b7bda7: done: 0.014768 sec","job_status":"done","queue_duration_s":1.883544,"scheduling_latency_s":1.883565,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014768,"completed_at":"2026-07-09T06:52:09.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1701gitlab # [ 204.092959] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"543d423a460785617298f5bd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-543d423a460785617298f5bd: start","job_status":"start","queue_duration_s":1.899539,"scheduling_latency_s":1.899559}1702gitlab # [ 204.093963] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"543d423a460785617298f5bd","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-543d423a460785617298f5bd: done: 0.014623 sec","job_status":"done","queue_duration_s":1.899539,"scheduling_latency_s":1.899559,"redis_calls":13,"redis_duration_s":0.002814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014623,"completed_at":"2026-07-09T06:52:09.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1703gitlab # [ 204.165052] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bd754168800b3468f2b8930","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bd754168800b3468f2b8930: start","job_status":"start","queue_duration_s":1.915394,"scheduling_latency_s":1.915415}1704gitlab # [ 204.166729] gitaly[1397]: time="2026-07-09T06:52:08.207Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4952 command.oublock=8 command.real_time_ms=50 command.spawn_token_fork_ms=0 command.system_time_ms=38 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:18.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.148" grpc.time_ms=58.529 pid=1397 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1705gitlab # [ 204.236459] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bd754168800b3468f2b8930","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bd754168800b3468f2b8930: done: 0.014681 sec","job_status":"done","queue_duration_s":1.915394,"scheduling_latency_s":1.915415,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-07-09T06:52:09.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1706gitlab # [ 204.238711] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1776d243705de3b2ffe43773","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1776d243705de3b2ffe43773: start","job_status":"start","queue_duration_s":1.93131,"scheduling_latency_s":1.931329}1707gitlab # [ 204.306156] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1776d243705de3b2ffe43773","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1776d243705de3b2ffe43773: done: 0.014983 sec","job_status":"done","queue_duration_s":1.93131,"scheduling_latency_s":1.931329,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-07-09T06:52:09.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1708gitlab # [ 204.317048] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab7b0fd88abd7f971c1436f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab7b0fd88abd7f971c1436f: start","job_status":"start","queue_duration_s":1.947537,"scheduling_latency_s":1.947558}1709gitlab # [ 204.336440] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab7b0fd88abd7f971c1436f","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab7b0fd88abd7f971c1436f: done: 0.017686 sec","job_status":"done","queue_duration_s":1.947537,"scheduling_latency_s":1.947558,"redis_calls":13,"redis_duration_s":0.0032120000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017686,"completed_at":"2026-07-09T06:52:09.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1710gitlab # [ 204.438607] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dbd27e6aa2be018c96ad2c6","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dbd27e6aa2be018c96ad2c6: start","job_status":"start","queue_duration_s":1.966722,"scheduling_latency_s":1.966749}1711gitlab # [ 204.439925] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dbd27e6aa2be018c96ad2c6","created_at":"2026-07-09T06:52:07.535Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-5e486d6d866a4038","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9dda94dca7164241894803a4b6a8874","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.598Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dbd27e6aa2be018c96ad2c6: done: 0.015544 sec","job_status":"done","queue_duration_s":1.966722,"scheduling_latency_s":1.966749,"redis_calls":13,"redis_duration_s":0.002966,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015544,"completed_at":"2026-07-09T06:52:09.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1712gitlab # [ 204.533113] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.582Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"39326dbfe6bdc1adb746f769","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-39326dbfe6bdc1adb746f769: start","job_status":"start","queue_duration_s":1.642012,"scheduling_latency_s":1.642031}1713gitlab # [ 204.534350] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.592Z","message":"No existing merge request to be cleaned up."}1714gitlab # [ 204.534873] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.599Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"39326dbfe6bdc1adb746f769","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-39326dbfe6bdc1adb746f769: done: 0.016748 sec","job_status":"done","queue_duration_s":1.642012,"scheduling_latency_s":1.642031,"redis_calls":10,"redis_duration_s":0.00231,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001422,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016748,"completed_at":"2026-07-09T06:52:09.599Z","load_balancing_strategy":"primary","db_duration_s":0.002262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1715gitlab # [ 204.616232] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.600Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5fd31ef7f8bde1177be87b00","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5fd31ef7f8bde1177be87b00: start","job_status":"start","queue_duration_s":1.660064,"scheduling_latency_s":1.660084}1716gitlab # [ 204.617702] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.609Z","message":"No existing merge request to be cleaned up."}1717gitlab # [ 204.618811] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.615Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5fd31ef7f8bde1177be87b00","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5fd31ef7f8bde1177be87b00: done: 0.015128 sec","job_status":"done","queue_duration_s":1.660064,"scheduling_latency_s":1.660084,"redis_calls":10,"redis_duration_s":0.002193,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001309,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015128,"completed_at":"2026-07-09T06:52:09.615Z","load_balancing_strategy":"primary","db_duration_s":0.001496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1718gitlab # [ 204.685189] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.616Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e5b7ce3c5a6d0db42c12ae8f","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e5b7ce3c5a6d0db42c12ae8f: start","job_status":"start","queue_duration_s":1.676564,"scheduling_latency_s":1.676584}1719gitlab # [ 204.686714] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.626Z","message":"No existing merge request to be cleaned up."}1720gitlab # [ 204.687777] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.632Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e5b7ce3c5a6d0db42c12ae8f","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e5b7ce3c5a6d0db42c12ae8f: done: 0.015321 sec","job_status":"done","queue_duration_s":1.676564,"scheduling_latency_s":1.676584,"redis_calls":10,"redis_duration_s":0.002186,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015321,"completed_at":"2026-07-09T06:52:09.632Z","load_balancing_strategy":"primary","db_duration_s":0.00149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1721gitlab # [ 204.759128] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.633Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55fb4781e7fa87f8d957dfb0","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55fb4781e7fa87f8d957dfb0: start","job_status":"start","queue_duration_s":1.693511,"scheduling_latency_s":1.693532}1722gitlab # [ 204.760369] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.643Z","message":"No existing merge request to be cleaned up."}1723gitlab # [ 204.761210] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.649Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55fb4781e7fa87f8d957dfb0","created_at":"2026-07-09T06:52:07.934Z","trace_propagation_headers":{"sentry-trace":"8199a6e368b341a9b838f7b8ae0d30a3-e32acb221e92432d","baggage":"sentry-trace_id=8199a6e368b341a9b838f7b8ae0d30a3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"670f5921184dba8ac2333f11f81b83bd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:07.940Z","job_size_bytes":2,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55fb4781e7fa87f8d957dfb0: done: 0.015562 sec","job_status":"done","queue_duration_s":1.693511,"scheduling_latency_s":1.693532,"redis_calls":10,"redis_duration_s":0.002247,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015562,"completed_at":"2026-07-09T06:52:09.649Z","load_balancing_strategy":"primary","db_duration_s":0.001828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1724gitlab # [ 204.849482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.650Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"198bc18d10ce8e7cb17ca34b","created_at":"2026-07-09T06:52:08.110Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.114Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-198bc18d10ce8e7cb17ca34b: start","job_status":"start","queue_duration_s":1.536815,"scheduling_latency_s":1.536836}1725gitlab # [ 204.850684] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.729Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"198bc18d10ce8e7cb17ca34b","created_at":"2026-07-09T06:52:08.110Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.114Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-198bc18d10ce8e7cb17ca34b: done: 0.078945 sec","job_status":"done","queue_duration_s":1.536815,"scheduling_latency_s":1.536836,"gitaly_calls":2,"gitaly_duration_s":0.01412,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007125,"redis_read_bytes":68,"redis_write_bytes":4889,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001296,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000578,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001589,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002664,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340266,"duration_s":0.078945,"completed_at":"2026-07-09T06:52:09.729Z","load_balancing_strategy":"primary","db_duration_s":0.010362,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1726gitlab # [ 204.949986] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.731Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ee8ea92ce7eb940d0d401dac","created_at":"2026-07-09T06:52:08.123Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.127Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ee8ea92ce7eb940d0d401dac: start","job_status":"start","queue_duration_s":1.604516,"scheduling_latency_s":1.604538}1727gitlab # [ 204.951248] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.808Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ee8ea92ce7eb940d0d401dac","created_at":"2026-07-09T06:52:08.123Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.127Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ee8ea92ce7eb940d0d401dac: done: 0.076464 sec","job_status":"done","queue_duration_s":1.604516,"scheduling_latency_s":1.604538,"gitaly_calls":1,"gitaly_duration_s":0.026591,"redis_calls":15,"redis_duration_s":0.0034460000000000003,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000608,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.054819256999999996,"duration_s":0.076464,"completed_at":"2026-07-09T06:52:09.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1728gitlab # [ 205.032059] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.809Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"6d8e988e50b5eca74e2095af","created_at":"2026-07-09T06:52:08.218Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.220Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6d8e988e50b5eca74e2095af: start","job_status":"start","queue_duration_s":1.589656,"scheduling_latency_s":1.589677}1729gitlab # [ 205.033394] gitaly[1397]: time="2026-07-09T06:52:08.222Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-09T06:52:18.142" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.222" grpc.time_ms=0.22 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730gitlab # [ 205.034482] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.846Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"6d8e988e50b5eca74e2095af","created_at":"2026-07-09T06:52:08.218Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.220Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6d8e988e50b5eca74e2095af: done: 0.036723 sec","job_status":"done","queue_duration_s":1.589656,"scheduling_latency_s":1.589677,"gitaly_calls":1,"gitaly_duration_s":0.008379,"redis_calls":16,"redis_duration_s":0.003767,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000267,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001735,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036723,"completed_at":"2026-07-09T06:52:09.846Z","load_balancing_strategy":"primary","db_duration_s":0.002273,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1731gitlab # [ 205.123966] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:09.847Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2121f21a4cfd0b1f0088fcee","created_at":"2026-07-09T06:52:08.662Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ed5beec3d0aaa88c0a7bcc86d9ed83ae221a756799c3e091b3cbde973ac66c72","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.666Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2121f21a4cfd0b1f0088fcee: start","job_status":"start","queue_duration_s":1.18131,"scheduling_latency_s":1.181332}1732gitlab # [ 205.125191] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:10.588Z","class":"CreatePipelineWorker","correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"8cb378872b8fe1f7cde149c561f5802ee16ad182","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"8cb378872b8fe1f7cde149c561f5802ee16ad182","ref":"refs/heads/master"},"retry":0}1733gitlab # [ 205.198079] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.593Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2121f21a4cfd0b1f0088fcee","created_at":"2026-07-09T06:52:08.662Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-7e7d8d3a71af4c7a","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ed5beec3d0aaa88c0a7bcc86d9ed83ae221a756799c3e091b3cbde973ac66c72","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:08.666Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2121f21a4cfd0b1f0088fcee: done: 0.745913 sec","job_status":"done","queue_duration_s":1.18131,"scheduling_latency_s":1.181332,"gitaly_calls":7,"gitaly_duration_s":0.052917,"redis_calls":53,"redis_duration_s":0.009913,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.000862,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004541,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002368,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.622462,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000326298,"duration_s":0.745913,"completed_at":"2026-07-09T06:52:10.593Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050387,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1734gitlab # [ 205.273499] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.595Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1ea2965a9bc3648188c0ce9d","created_at":"2026-07-09T06:52:09.714Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-543eda3ffbc2465e","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:09.719Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1ea2965a9bc3648188c0ce9d: start","job_status":"start","queue_duration_s":0.876007,"scheduling_latency_s":0.876029}1735gitlab # [ 205.274624] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.638Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1ea2965a9bc3648188c0ce9d","created_at":"2026-07-09T06:52:09.714Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-543eda3ffbc2465e","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:09.719Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1ea2965a9bc3648188c0ce9d: done: 0.043032 sec","job_status":"done","queue_duration_s":0.876007,"scheduling_latency_s":0.876029,"redis_calls":19,"redis_duration_s":0.004334,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00096,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00064,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001998,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006407797,"duration_s":0.043032,"completed_at":"2026-07-09T06:52:10.638Z","load_balancing_strategy":"primary","db_duration_s":0.005534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1736gitlab # [ 205.305158] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.639Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c88ce8773b167a5659566ebc","created_at":"2026-07-09T06:52:10.627Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-60d8021c7cc14146","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:10.631Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c88ce8773b167a5659566ebc: start","job_status":"start","queue_duration_s":0.008241,"scheduling_latency_s":0.008261}1737gitlab # [ 205.376139] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:10.701Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c88ce8773b167a5659566ebc","created_at":"2026-07-09T06:52:10.627Z","trace_propagation_headers":{"sentry-trace":"921dd8203334415cba7f1553af81101a-60d8021c7cc14146","baggage":"sentry-trace_id=921dd8203334415cba7f1553af81101a,sentry-release=Unknown"},"correlation_id":"01KX2TBRC1HYYKEKYSGC4SRJDV","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:10.631Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c88ce8773b167a5659566ebc: done: 0.062116 sec","job_status":"done","queue_duration_s":0.008241,"scheduling_latency_s":0.008261,"redis_calls":11,"redis_duration_s":0.00265,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000331606,"duration_s":0.062116,"completed_at":"2026-07-09T06:52:10.701Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1738gitlab # [ 205.451699] gitaly[1397]: time="2026-07-09T06:52:08.235Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-09T06:52:18.146" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.226" grpc.time_ms=9.316 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab # [ 205.453005] gitaly[1397]: time="2026-07-09T06:52:08.295Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-09T06:52:38.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:08.289" grpc.time_ms=5.255 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740gitlab # [ 205.454075] gitaly[1397]: time="2026-07-09T06:52:08.316Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:18.232" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:08.312" grpc.time_ms=0.203 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 205.458245] gitaly[1397]: time="2026-07-09T06:52:08.324Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-09T06:52:18.239" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-09T06:52:08.320" grpc.time_ms=4.387 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 205.536221] gitaly[1397]: time="2026-07-09T06:52:08.343Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:18.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.336" grpc.time_ms=6.529 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 205.537410] gitaly[1397]: time="2026-07-09T06:52:08.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:18.266" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:08.347" grpc.time_ms=6.294 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 205.537948] gitaly[1397]: time="2026-07-09T06:52:08.371Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:38.161" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:08.361" grpc.time_ms=9.636 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 205.542145] gitaly[1397]: time="2026-07-09T06:52:08.690Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-09T06:52:18.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:08.689" grpc.time_ms=1.689 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 205.608192] gitaly[1397]: time="2026-07-09T06:52:09.689Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-09T12:52:09.686" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:09.687" grpc.time_ms=2.355 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747gitlab # [ 205.609408] gitaly[1397]: time="2026-07-09T06:52:09.705Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-09T06:52:39.497" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.698" grpc.time_ms=6.787 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1748gitlab # [ 205.609892] gitaly[1397]: time="2026-07-09T06:52:09.776Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV diskcache=968ea131-e424-49d0-b9b9-ebcee1bc787b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:09.752" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.752" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1749gitlab # [ 205.612418] gitaly[1397]: time="2026-07-09T06:52:09.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1284 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:09.752" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.752" grpc.time_ms=24.055 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1750gitlab # [ 205.680237] gitaly[1397]: time="2026-07-09T06:52:09.839Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-09T06:52:39.632" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.832" grpc.time_ms=6.087 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751gitlab # [ 205.681809] gitaly[1397]: time="2026-07-09T06:52:09.882Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:39.680" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.881" grpc.time_ms=0.708 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752gitlab # [ 205.682864] gitaly[1397]: time="2026-07-09T06:52:09.891Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:19.204" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:09.884" grpc.time_ms=6.755 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 205.751374] gitaly[1397]: time="2026-07-09T06:52:09.974Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:39.772" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.972" grpc.time_ms=1.719 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754gitlab # [ 205.752880] gitaly[1397]: time="2026-07-09T06:52:09.980Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=194 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-09T06:52:39.777" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:09.977" grpc.time_ms=2.592 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755gitlab # [ 205.753686] gitaly[1397]: time="2026-07-09T06:52:10.055Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:20.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:10.048" grpc.time_ms=6.107 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 205.758189] gitaly[1397]: time="2026-07-09T06:52:10.546Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:40.334" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:10.535" grpc.time_ms=10.747 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757gitlab # [ 205.782179] gitaly[1397]: time="2026-07-09T06:52:10.557Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TBRC1HYYKEKYSGC4SRJDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-09T06:52:40.348" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:10.548" grpc.time_ms=9.189 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab # [ 206.128341] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6549269477 wd_nsec: 65492694481759gitlab # [ 206.817647] nixbot[3109]: {"ts": "2026-07-09T06:52:15.620Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1760gitlab # [ 206.837489] nixbot[3109]: {"ts": "2026-07-09T06:52:15.640Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1761gitlab # [ 206.868697] nixbot[3109]: {"ts": "2026-07-09T06:52:15.671Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1762gitlab # [ 206.875133] nixbot[3109]: {"ts": "2026-07-09T06:52:15.678Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1763gitlab # [ 206.881659] nixbot[3109]: {"ts": "2026-07-09T06:52:15.684Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1764gitlab # [ 206.905775] nixbot[3109]: {"ts": "2026-07-09T06:52:15.708Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1765gitlab # [ 206.912120] nixbot[3109]: {"ts": "2026-07-09T06:52:15.715Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1766gitlab # [ 206.916153] nixbot[3109]: {"ts": "2026-07-09T06:52:15.719Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1767gitlab # [ 206.925735] nixbot[3109]: {"ts": "2026-07-09T06:52:15.728Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1768gitlab # [ 206.929804] nixbot[3109]: {"ts": "2026-07-09T06:52:15.732Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1769gitlab # [ 206.938167] nixbot[3109]: {"ts": "2026-07-09T06:52:15.741Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1770gitlab # [ 206.948197] nixbot[3109]: {"ts": "2026-07-09T06:52:15.751Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1771gitlab # [ 206.954943] nixbot[3109]: {"ts": "2026-07-09T06:52:15.758Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1772gitlab # [ 206.959671] nixbot[3109]: {"ts": "2026-07-09T06:52:15.762Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1773gitlab # [ 206.966653] nixbot[3109]: {"ts": "2026-07-09T06:52:15.769Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1774gitlab # [ 206.970947] nixbot[3109]: {"ts": "2026-07-09T06:52:15.774Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1775gitlab # [ 206.974840] nixbot[3109]: {"ts": "2026-07-09T06:52:15.778Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1776gitlab # [ 206.978834] nixbot[3109]: {"ts": "2026-07-09T06:52:15.782Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1777gitlab # [ 206.986494] nixbot[3109]: {"ts": "2026-07-09T06:52:15.789Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1778gitlab # [ 206.990490] nixbot[3109]: {"ts": "2026-07-09T06:52:15.793Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1779gitlab # [ 206.994642] nixbot[3109]: {"ts": "2026-07-09T06:52:15.797Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1780gitlab # [ 207.325640] nixbot[3109]: INFO: Started server process [3109]1781gitlab # [ 207.329549] nixbot[3109]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1782gitlab # [ 207.333418] nixbot[3109]: INFO: ::ffff:127.0.0.1:53500 - "GET /health HTTP/1.1" 200 OK1783gitlab # [ 207.341714] systemd[1]: Started nixbot CI service.1784gitlab # [ 207.344714] systemd[1]: Reached target Multi-User System.1785gitlab # [ 207.347979] systemd[1]: Startup finished in 1.609s (kernel) + 4.461s (initrd) + 3min 21.274s (userspace) = 3min 27.344s.1786gitlab # [ 207.560810] gitaly[1397]: time="2026-07-09T06:52:16.363Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC1C5PZ1JB18F4EZTZK8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:46.242" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:16.343" grpc.time_ms=19.845 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 207.662975] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 3311788gitlab # [ 207.665974] nixbot[3109]: {"ts": "2026-07-09T06:52:16.467Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1789gitlab # [ 207.696418] nixbot[3109]: {"ts": "2026-07-09T06:52:16.499Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1790gitlab # [ 207.855153] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1441791gitlab # [ 207.865886] nixbot[3109]: {"ts": "2026-07-09T06:52:16.668Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1792gitlab # [ 207.876710] nixbot[3109]: {"ts": "2026-07-09T06:52:16.679Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1793gitlab # [ 208.015097] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1271794gitlab # [ 208.024511] nixbot[3109]: {"ts": "2026-07-09T06:52:16.827Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1795gitlab # [ 208.166256] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:16 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1191796gitlab # [ 208.168983] nixbot[3109]: {"ts": "2026-07-09T06:52:16.970Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1797gitlab # [ 208.238430] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 441798gitlab # [ 208.241223] nixbot[3109]: {"ts": "2026-07-09T06:52:17.042Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1799gitlab # [ 208.287232] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 441800gitlab # [ 208.289976] nixbot[3109]: {"ts": "2026-07-09T06:52:17.090Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1801gitlab: (finished: waiting for unit nixbot.service, in 9.29 seconds)1802gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1803gitlab # [ 208.406347] gitaly[1397]: time="2026-07-09T06:52:17.209Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2AR2J7TF9Q68MB8E4DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-09T06:52:47.091" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:17.192" grpc.time_ms=15.918 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 208.424201] gitaly[1397]: time="2026-07-09T06:52:17.227Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2AR2J7TF9Q68MB8E4DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:47.124" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:17.225" grpc.time_ms=1.049 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 208.481916] gitaly[1397]: time="2026-07-09T06:52:17.285Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2AR2J7TF9Q68MB8E4DJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-09T06:52:27.010" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:17.231" grpc.time_ms=53.713 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 208.496867] nixbot[3109]: INFO: ::ffff:127.0.0.1:53504 - "GET /health HTTP/1.1" 200 OK1807gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.11 seconds)1808subtest: project discovered and webhook registered1809gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1810gitlab # [ 208.552928] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 2431811gitlab # [ 208.556974] nixbot[3109]: {"ts": "2026-07-09T06:52:17.357Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1812gitlab # [ 208.608625] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 801813gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1814(finished: subtest: project discovered and webhook registered, in 0.11 seconds)1815subtest: push delivers the webhook1816gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1817gitlab # [ 208.675500] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 231818gitlab # [ 208.684697] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1251819gitlab # [ 208.695872] nixbot[3109]: {"ts": "2026-07-09T06:52:17.498Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1820gitlab # [ 208.709386] nixbot[3109]: {"ts": "2026-07-09T06:52:17.512Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1821gitlab # [ 208.744275] gitaly[1397]: time="2026-07-09T06:52:17.546Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2PFSABP39MNKSPPT2V4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.537" grpc.time_ms=8.858 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1822gitlab # [ 208.746235] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 611823gitlab # Enumerating objects: 5, done.1824gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1825gitlab # Delta compression using up to 4 threads1826gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1827gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1828gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1829gitlab # [ 208.833732] gitaly[1397]: time="2026-07-09T06:52:17.631Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2S5X7D3XTS103B7330W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.615" grpc.time_ms=15.053 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1830gitlab # [ 208.835044] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 601831gitlab # [ 208.946376] gitaly[1397]: time="2026-07-09T06:52:17.744Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2V49RXTEEKEE0N1ZRA4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.730" pid=1397 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1832gitlab # [ 208.947982] gitlab-workhorse[1306]: time="2026-07-09T06:52:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491833gitlab # [ 208.949868] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1101834gitlab # [ 208.951105] gitaly[1397]: time="2026-07-09T06:52:17.744Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2V49RXTEEKEE0N1ZRA4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.730" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1835gitlab # [ 208.953049] gitaly[1397]: time="2026-07-09T06:52:17.745Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2V49RXTEEKEE0N1ZRA4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.730" grpc.time_ms=14.336 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1836gitlab # [ 209.031178] gitaly[1397]: time="2026-07-09T06:52:17.834Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2YRJ9Q48ZRS13X4QS5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.817" pid=1397 remote_ip="2001:db8:1::1" request_sha=87147561dfef65e1b2b7d6831dfe8d56f471701b response_bytes=229 span.kind=server system=grpc1837gitlab # [ 209.034889] gitaly[1397]: time="2026-07-09T06:52:17.836Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2YRJ9Q48ZRS13X4QS5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.817" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1838gitlab # [ 209.036208] gitlab-workhorse[1306]: time="2026-07-09T06:52:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2291839gitlab # [ 209.036993] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 229 "" "git/2.54.0" 841840gitlab # [ 209.037898] gitaly[1397]: time="2026-07-09T06:52:17.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2YRJ9Q48ZRS13X4QS5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.817" grpc.time_ms=19.069 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1841gitlab # [ 209.156776] gitaly[1397]: time="2026-07-09T06:52:17.959Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-09T06:52:27.001" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:17.952" grpc.time_ms=7.153 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 209.184224] gitaly[1397]: time="2026-07-09T06:52:17.987Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=333 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-09T06:52:47.876" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-09T06:52:17.976" grpc.time_ms=11.107 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 209.197102] gitaly[1397]: time="2026-07-09T06:52:17.999Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC34HQ6D87B4JJFCFZGVG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.980" grpc.time_ms=19.277 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1844gitlab # [ 209.198084] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 641845gitlab # [ 209.201556] gitaly[1397]: time="2026-07-09T06:52:18.004Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 duration_ms=99ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/allowed"1846gitlab # [ 209.229567] gitaly[1397]: time="2026-07-09T06:52:18.033Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 duration_ms=27ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/pre_receive"1847gitlab # [ 209.231962] gitaly[1397]: time="2026-07-09T06:52:18.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:17.904" grpc.time_ms=131.064 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1848gitlab # [ 209.263615] gitaly[1397]: time="2026-07-09T06:52:18.067Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC36MVQ8YDY1V4C049FBJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.057" pid=1397 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1849gitlab # [ 209.265074] gitaly[1397]: time="2026-07-09T06:52:18.067Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC36MVQ8YDY1V4C049FBJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.057" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1850gitlab # [ 209.275629] gitaly[1397]: time="2026-07-09T06:52:18.078Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC36MVQ8YDY1V4C049FBJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.057" grpc.time_ms=21.196 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1851gitlab # [ 209.277333] gitlab-workhorse[1306]: time="2026-07-09T06:52:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671852gitlab # [ 209.278763] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 751853gitlab # [ 209.314961] gitaly[1397]: time="2026-07-09T06:52:18.118Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.116" grpc.time_ms=1.73 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1854gitlab # [ 209.370108] gitaly[1397]: time="2026-07-09T06:52:18.173Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.172" grpc.time_ms=0.56 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1855gitlab # [ 209.410607] gitaly[1397]: time="2026-07-09T06:52:18.214Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.213" grpc.time_ms=0.434 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1856gitlab # [ 209.443401] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 701857gitlab # [ 209.444841] gitaly[1397]: time="2026-07-09T06:52:18.245Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3BZG692AFAAPCY0EZ94 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.232" grpc.time_ms=12.516 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1858gitlab # [ 209.491349] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.294Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8cb378872b8fe1f7cde149c561f5802ee16ad182 4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b7f6af44dfa26baeb21cf232","created_at":"2026-07-09T06:52:18.290Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-ec2b0fb054e04464","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.292Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b7f6af44dfa26baeb21cf232: start","job_status":"start","queue_duration_s":0.001737,"scheduling_latency_s":0.001776}1859gitlab # [ 209.512686] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.315Z","message":"changes: \"8cb378872b8fe1f7cde149c561f5802ee16ad182 4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf refs/heads/master\\n\""}1860gitlab # [ 209.519345] gitaly[1397]: time="2026-07-09T06:52:18.322Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3EAKHNFCV6DF786WZS5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.307" pid=1397 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc1861gitlab # [ 209.521441] gitaly[1397]: time="2026-07-09T06:52:18.322Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3EAKHNFCV6DF786WZS5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.307" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1862gitlab # [ 209.522402] gitlab-workhorse[1306]: time="2026-07-09T06:52:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2141863gitlab # [ 209.523701] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 731864gitlab # [ 209.525106] gitaly[1397]: time="2026-07-09T06:52:18.323Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3EAKHNFCV6DF786WZS5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:18.307" grpc.time_ms=16.002 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1865gitlab # [ 209.527092] gitaly[1397]: time="2026-07-09T06:52:18.330Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 duration_ms=64ns method=POST pid=1397 status=200 url="http://unix/api/v4/internal/post_receive"1866gitlab # [ 209.529307] gitaly[1397]: time="2026-07-09T06:52:18.332Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:18.265" grpc.time_ms=66.903 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1867gitlab # [ 209.546249] gitaly[1397]: time="2026-07-09T06:52:18.349Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:28.260" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.341" grpc.time_ms=8.318 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 209.560663] gitaly[1397]: time="2026-07-09T06:52:18.361Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-09T06:52:28.273" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.354" grpc.time_ms=7.518 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869gitlab # [ 209.637553] gitaly[1397]: time="2026-07-09T06:52:18.385Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-09T06:52:28.295" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.375" grpc.time_ms=9.717 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870gitlab # [ 209.639108] gitaly[1397]: time="2026-07-09T06:52:18.402Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-09T06:52:48.189" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.390" grpc.time_ms=12.169 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1871gitlab # [ 209.648232] gitaly[1397]: time="2026-07-09T06:52:18.451Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 diskcache=b5e09037-b030-48a4-90ea-c059e1c9ad40 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.699" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1872gitlab # [ 209.650128] gitaly[1397]: time="2026-07-09T06:52:18.453Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=261 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=12453 command.oublock=32 command.real_time_ms=736 command.spawn_token_fork_ms=2 command.system_time_ms=215 command.user_time_ms=46 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:17.699" grpc.time_ms=753.438 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1873gitlab # [ 209.718933] gitlab-workhorse[1306]: time="2026-07-09T06:52:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521874gitlab # [ 209.719661] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:18 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 9261875gitlab # To http://gitlab/root/test-flake.git1876gitlab # 8cb3788..4628e24 master -> master1877gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.13 seconds)1878gitlab: must succeed: git -C /tmp/test-flake rev-parse master1879gitlab # [ 209.741851] gitaly[1397]: time="2026-07-09T06:52:18.545Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-09T06:52:28.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-09T06:52:18.438" grpc.time_ms=106.504 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1881gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf'1882gitlab # [ 209.764377] gitaly[1397]: time="2026-07-09T06:52:18.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:28.125" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.555" grpc.time_ms=10.524 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab # [ 209.811699] nixbot[3109]: INFO: ::ffff:127.0.0.1:53508 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 200 OK1884gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', in 0.05 seconds)1885gitlab # [ 209.843371] gitaly[1397]: time="2026-07-09T06:52:18.646Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:48.545" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.645" grpc.time_ms=0.659 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886gitlab # [ 209.892511] gitaly[1397]: time="2026-07-09T06:52:18.695Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-09T06:52:48.588" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.688" grpc.time_ms=7.092 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 209.920993] gitaly[1397]: time="2026-07-09T06:52:18.724Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:28.247" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.718" grpc.time_ms=6.105 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 209.958413] gitaly[1397]: time="2026-07-09T06:52:18.761Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-09T06:52:28.079" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.759" grpc.time_ms=1.683 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 209.986981] gitaly[1397]: time="2026-07-09T06:52:18.790Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:28.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.789" grpc.time_ms=0.23 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 209.999699] gitaly[1397]: time="2026-07-09T06:52:18.803Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=618 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:28.016" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.797" grpc.time_ms=5.823 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 210.036618] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.839Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8cb378872b8fe1f7cde149c561f5802ee16ad182 4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b7f6af44dfa26baeb21cf232","created_at":"2026-07-09T06:52:18.290Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-ec2b0fb054e04464","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.292Z","job_size_bytes":185,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b7f6af44dfa26baeb21cf232: done: 0.545449 sec","job_status":"done","queue_duration_s":0.001737,"scheduling_latency_s":0.001776,"gitaly_calls":7,"gitaly_duration_s":0.201288,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.072283,"redis_read_bytes":350,"redis_write_bytes":13068,"redis_cache_calls":9,"redis_cache_duration_s":0.004662,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.020869,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001217,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003074,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.01204,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.030421,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.196576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002268445,"exclusive_lock_hold_duration_s":0.022666568,"duration_s":0.545449,"completed_at":"2026-07-09T06:52:18.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035697,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1892gitlab # [ 210.048351] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.844Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1a5ce2f5cb4330a03290f3da","created_at":"2026-07-09T06:52:18.363Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.366Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1a5ce2f5cb4330a03290f3da: start","job_status":"start","queue_duration_s":0.478072,"scheduling_latency_s":0.478104}1893gitlab # [ 210.125987] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.879Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1a5ce2f5cb4330a03290f3da","created_at":"2026-07-09T06:52:18.363Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.366Z","job_size_bytes":46,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1a5ce2f5cb4330a03290f3da: done: 0.035183 sec","job_status":"done","queue_duration_s":0.478072,"scheduling_latency_s":0.478104,"redis_calls":9,"redis_duration_s":0.0053029999999999996,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000301,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002886,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035183,"completed_at":"2026-07-09T06:52:18.879Z","load_balancing_strategy":"primary","db_duration_s":0.002437,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1894gitlab # [ 210.197159] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.881Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b8c955a3e94fa05cbe47b2c1","created_at":"2026-07-09T06:52:18.370Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.372Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b8c955a3e94fa05cbe47b2c1: start","job_status":"start","queue_duration_s":0.509668,"scheduling_latency_s":0.509695}1895gitlab # [ 210.198569] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.954Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b8c955a3e94fa05cbe47b2c1","created_at":"2026-07-09T06:52:18.370Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.372Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b8c955a3e94fa05cbe47b2c1: done: 0.073087 sec","job_status":"done","queue_duration_s":0.509668,"scheduling_latency_s":0.509695,"gitaly_calls":1,"gitaly_duration_s":0.038748,"redis_calls":12,"redis_duration_s":0.004096,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000903,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055107563,"duration_s":0.073087,"completed_at":"2026-07-09T06:52:18.954Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1896gitlab # [ 210.291119] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:18.956Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f1781817544c4778c19f29df","created_at":"2026-07-09T06:52:18.623Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6d6e9817cc614eeaa9f3f9053a817f1e4d7146d29c31755c8f68eebea7c941d1","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.626Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f1781817544c4778c19f29df: start","job_status":"start","queue_duration_s":0.329554,"scheduling_latency_s":0.329576}1897gitlab # [ 210.292457] gitaly[1397]: time="2026-07-09T06:52:18.935Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 diskcache=1bb3a2ca-fc1f-4a0f-97ee-cc14d7de48d0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:18.899" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.899" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 210.293753] gitaly[1397]: time="2026-07-09T06:52:18.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1646 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-09T12:52:18.899" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.899" grpc.time_ms=36.176 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 210.364199] gitaly[1397]: time="2026-07-09T06:52:18.978Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:48.777" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:18.978" grpc.time_ms=0.575 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 210.365656] gitaly[1397]: time="2026-07-09T06:52:18.986Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:28.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:18.980" grpc.time_ms=6.297 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 210.367124] gitaly[1397]: time="2026-07-09T06:52:19.013Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:49.012" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.013" grpc.time_ms=0.685 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902gitlab # [ 210.367993] gitaly[1397]: time="2026-07-09T06:52:19.020Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=321 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-09T06:52:49.016" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.016" grpc.time_ms=3.49 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 210.384092] gitaly[1397]: time="2026-07-09T06:52:19.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:29.118" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:19.158" grpc.time_ms=6.684 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 210.474624] gitaly[1397]: time="2026-07-09T06:52:19.278Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:19.276" grpc.time_ms=0.913 pid=1397 span.kind=server system=grpc1905gitlab # [ 210.514290] gitaly[1397]: time="2026-07-09T06:52:19.317Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:19.316" grpc.time_ms=0.774 pid=1397 span.kind=server system=grpc1906gitlab # [ 210.538762] gitaly[1397]: time="2026-07-09T06:52:19.342Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G diskcache=1fffc81f-cda8-4f4c-9aa6-a00f06243cb6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:28.024" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.805" pid=1397 ref=refs/keep-around/8cb378872b8fe1f7cde149c561f5802ee16ad182 remote_ip="2001:db8:1::1" revision=8cb378872b8fe1f7cde149c561f5802ee16ad182 span.kind=server system=grpc username=root1907gitlab # [ 210.539881] gitaly[1397]: time="2026-07-09T06:52:19.342Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=264 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5411 command.oublock=8 command.real_time_ms=526 command.spawn_token_fork_ms=0 command.system_time_ms=201 command.user_time_ms=63 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:28.024" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:18.805" grpc.time_ms=537.133 pid=1397 ref=refs/keep-around/8cb378872b8fe1f7cde149c561f5802ee16ad182 remote_ip="2001:db8:1::1" revision=8cb378872b8fe1f7cde149c561f5802ee16ad182 span.kind=server system=grpc username=root1908gitlab # [ 210.544729] gitaly[1397]: time="2026-07-09T06:52:19.348Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:49.247" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.347" grpc.time_ms=0.422 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 210.666293] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 8751910gitlab # [ 210.671515] nixbot[3109]: {"ts": "2026-07-09T06:52:19.473Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 \"HTTP/1.1 201 Created\""}1911gitlab # [ 210.697889] nixbot[3109]: {"ts": "2026-07-09T06:52:19.501Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-8cb378872b8f-2dcca477", "argv0": "bwrap"}1912gitlab # [ 210.805964] systemd[1]: Started Nix Daemon.1913gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf'1914gitlab # [ 210.848940] nixbot[3109]: INFO: ::ffff:127.0.0.1:53514 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 200 OK1915gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', in 0.04 seconds)1916gitlab # [ 210.862291] nix-daemon[3445]: accepted connection from pid 3444, user nixbot1917gitlab # [ 210.868527] nix-daemon[3445]: accepted connection from pid 3443, user nixbot1918gitlab # [ 210.879223] gitaly[1397]: time="2026-07-09T06:52:19.682Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=3 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-09T06:52:49.660" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.661" grpc.time_ms=21.137 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 210.924167] gitaly[1397]: time="2026-07-09T06:52:19.727Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC2SR77YMGK7M7MH2XEE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-09T06:52:49.685" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:19.686" grpc.time_ms=41.128 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 210.935501] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:19.738Z","class":"CreatePipelineWorker","correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"8cb378872b8fe1f7cde149c561f5802ee16ad182","after":"4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf","ref":"refs/heads/master"},"retry":0}1921gitlab # [ 210.940288] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.743Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f1781817544c4778c19f29df","created_at":"2026-07-09T06:52:18.623Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6d6e9817cc614eeaa9f3f9053a817f1e4d7146d29c31755c8f68eebea7c941d1","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.626Z","job_size_bytes":467,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f1781817544c4778c19f29df: done: 0.786944 sec","job_status":"done","queue_duration_s":0.329554,"scheduling_latency_s":0.329576,"gitaly_calls":7,"gitaly_duration_s":0.197708,"redis_calls":33,"redis_duration_s":0.011017,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001266,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.007353,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.104,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.101,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.563669,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.786944,"completed_at":"2026-07-09T06:52:19.743Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.103731,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1922gitlab # [ 211.012296] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.745Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"ad58b130-a013-449a-b541-d97ea263950d\"}"],"class":"WebHookWorker","jid":"bb3f4cbc1fee517d06f37724","created_at":"2026-07-09T06:52:18.722Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7b2c9ec3a9499bac32878a43d0ff93018e4fcbf7fdfa13a67501800a2384c11b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.749Z","job_size_bytes":1749,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bb3f4cbc1fee517d06f37724: start","job_status":"start","queue_duration_s":0.995459,"scheduling_latency_s":0.995484}1923gitlab # [ 211.103658] nixbot[3109]: INFO: ::ffff:127.0.0.1:53530 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1924gitlab # [ 211.125243] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.928Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"ad58b130-a013-449a-b541-d97ea263950d\"}"],"class":"WebHookWorker","jid":"bb3f4cbc1fee517d06f37724","created_at":"2026-07-09T06:52:18.722Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-dc81254dc8da49c0","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7b2c9ec3a9499bac32878a43d0ff93018e4fcbf7fdfa13a67501800a2384c11b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:18.749Z","job_size_bytes":1749,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bb3f4cbc1fee517d06f37724: done: 0.182999 sec","job_status":"done","queue_duration_s":0.995459,"scheduling_latency_s":0.995484,"redis_calls":19,"redis_duration_s":0.004783,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001339,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.00036,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00193,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.01138887799999111,"cpu_s":0.063528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384685,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.182999,"completed_at":"2026-07-09T06:52:19.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1925gitlab # [ 211.133431] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.929Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"81a1eedc830794b873face65","created_at":"2026-07-09T06:52:19.353Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.359Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-81a1eedc830794b873face65: start","job_status":"start","queue_duration_s":0.5703,"scheduling_latency_s":0.570323}1926gitlab # [ 211.209552] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.988Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"81a1eedc830794b873face65","created_at":"2026-07-09T06:52:19.353Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.359Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-81a1eedc830794b873face65: done: 0.058562 sec","job_status":"done","queue_duration_s":0.5703,"scheduling_latency_s":0.570323,"redis_calls":18,"redis_duration_s":0.010326,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004511,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000550628,"duration_s":0.058562,"completed_at":"2026-07-09T06:52:19.988Z","load_balancing_strategy":"primary","db_duration_s":0.014067,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1927gitlab # [ 211.228943] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:19.990Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f5dd9395c88859d729d759bc","created_at":"2026-07-09T06:52:19.360Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.363Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f5dd9395c88859d729d759bc: start","job_status":"start","queue_duration_s":0.626757,"scheduling_latency_s":0.62678}1928gitlab # [ 211.303053] gitaly[1397]: time="2026-07-09T06:52:20.103Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5368RAZ399238YQEY1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.092" grpc.time_ms=10.968 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1929gitlab # [ 211.304627] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1611930gitlab # [ 211.366811] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:20.169Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f5dd9395c88859d729d759bc","created_at":"2026-07-09T06:52:19.360Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.363Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f5dd9395c88859d729d759bc: done: 0.179129 sec","job_status":"done","queue_duration_s":0.626757,"scheduling_latency_s":0.62678,"redis_calls":15,"redis_duration_s":0.005597,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001123,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001788,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000404242,"duration_s":0.179129,"completed_at":"2026-07-09T06:52:20.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1931gitlab # [ 211.370472] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:20.173Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"56b6b01ed8fe7f6e2b7b38fe","created_at":"2026-07-09T06:52:19.448Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.454Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56b6b01ed8fe7f6e2b7b38fe: start","job_status":"start","queue_duration_s":0.718926,"scheduling_latency_s":0.71895}1932gitlab # [ 211.393225] gitaly[1397]: time="2026-07-09T06:52:20.181Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC589NWZTG1VWHVN725T6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.161" pid=1397 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1933gitlab # [ 211.467833] gitlab-workhorse[1306]: time="2026-07-09T06:52:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671934gitlab # [ 211.469274] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 771935gitlab # [ 211.471973] gitlab-workhorse[1306]: time="2026-07-09T06:52:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3411936gitlab # [ 211.474206] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 341 "" "git/2.54.0" 791937gitlab # [ 211.476437] gitaly[1397]: time="2026-07-09T06:52:20.182Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC589NWZTG1VWHVN725T6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.161" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1938gitlab # [ 211.477362] gitaly[1397]: time="2026-07-09T06:52:20.182Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC589NWZTG1VWHVN725T6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.161" grpc.time_ms=20.93 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1939gitlab # [ 211.479743] gitaly[1397]: time="2026-07-09T06:52:20.268Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5AYBMVE56907DXS4S5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.253" pid=1397 remote_ip="2001:db8:1::1" request_sha=48ab23ff9c165208b0619f9fed6b02fe185097b7 response_bytes=341 span.kind=server system=grpc1940gitlab # [ 211.480825] gitaly[1397]: time="2026-07-09T06:52:20.268Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5AYBMVE56907DXS4S5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.253" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1941gitlab # [ 211.548989] gitaly[1397]: time="2026-07-09T06:52:20.269Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5AYBMVE56907DXS4S5P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.253" grpc.time_ms=15.777 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1942gitlab # [ 211.677676] gitaly[1397]: time="2026-07-09T06:52:20.481Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5JHYHH4SR5NCR8MYJG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.471" grpc.time_ms=8.511 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1943gitlab # [ 211.700408] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 691944gitlab # [ 211.763132] gitaly[1397]: time="2026-07-09T06:52:20.566Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5MVA50QBCBYKDYA35ST grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.548" pid=1397 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc1945gitlab # [ 211.767438] gitaly[1397]: time="2026-07-09T06:52:20.568Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5MVA50QBCBYKDYA35ST grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.548" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1946gitlab # [ 211.768748] gitlab-workhorse[1306]: time="2026-07-09T06:52:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161947gitlab # [ 211.769810] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 611948gitlab # [ 211.770806] gitaly[1397]: time="2026-07-09T06:52:20.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5MVA50QBCBYKDYA35ST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-09T06:52:20.548" grpc.time_ms=20.485 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc1949gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf'1950gitlab # [ 211.845496] gitaly[1397]: time="2026-07-09T06:52:20.648Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:50.646" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.646" grpc.time_ms=1.79 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951gitlab # [ 211.884701] nixbot[3109]: {"ts": "2026-07-09T06:52:20.687Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1952gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', in 0.04 seconds)1953gitlab # [ 211.891067] nixbot[3109]: INFO: ::ffff:127.0.0.1:53538 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 200 OK1954(finished: subtest: push delivers the webhook, in 3.28 seconds)1955subtest: eval and build post commit statuses1956gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses1957gitlab # [ 211.892495] nix-daemon[3454]: unexpected Nix daemon error: error: interrupted by the user1958gitlab # [ 211.919096] gitaly[1397]: time="2026-07-09T06:52:20.722Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.720" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.721" grpc.time_ms=0.841 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1959gitlab # [ 211.939058] gitaly[1397]: time="2026-07-09T06:52:20.742Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.641" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.741" grpc.time_ms=0.483 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 211.964679] gitaly[1397]: time="2026-07-09T06:52:20.768Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5VKS0HGPEK3CJEDCR73 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.666" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.767" grpc.time_ms=0.621 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 211.975780] gitaly[1397]: time="2026-07-09T06:52:20.779Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-09T06:52:50.671" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:20.772" grpc.time_ms=6.715 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 211.989332] gitaly[1397]: time="2026-07-09T06:52:20.792Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:30.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:20.785" grpc.time_ms=7.55 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 212.009381] gitaly[1397]: time="2026-07-09T06:52:20.812Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-09T06:52:30.030" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:20.811" grpc.time_ms=0.199 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964gitlab # [ 212.020748] gitaly[1397]: time="2026-07-09T06:52:20.824Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=617 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-09T06:52:30.034" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:20.814" grpc.time_ms=9.172 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965gitlab # [ 212.067328] gitaly[1397]: time="2026-07-09T06:52:20.869Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:20.866" grpc.time_ms=2.87 pid=1397 span.kind=server system=grpc1966gitlab # [ 212.091136] gitaly[1397]: time="2026-07-09T06:52:20.894Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.793" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.893" grpc.time_ms=0.731 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 212.093770] gitaly[1397]: time="2026-07-09T06:52:20.897Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:20.896" grpc.time_ms=0.277 pid=1397 span.kind=server system=grpc1968gitlab # [ 212.136235] gitaly[1397]: time="2026-07-09T06:52:20.939Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ diskcache=80b6de0b-76dd-462f-ab07-13eb97b1465e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:30.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:20.828" pid=1397 ref=refs/keep-around/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf remote_ip="2001:db8:1::1" revision=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf span.kind=server system=grpc username=root1969gitlab # [ 212.154203] gitaly[1397]: time="2026-07-09T06:52:20.957Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=73 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5030 command.oublock=8 command.real_time_ms=100 command.spawn_token_fork_ms=2 command.system_time_ms=49 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-09T06:52:30.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-09T06:52:20.828" grpc.time_ms=111.467 pid=1397 ref=refs/keep-around/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf remote_ip="2001:db8:1::1" revision=4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf span.kind=server system=grpc username=root1970gitlab # [ 212.161090] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 2401971gitlab # [ 212.182401] gitaly[1397]: time="2026-07-09T06:52:20.985Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.884" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:20.985" grpc.time_ms=0.455 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.32 seconds)1973gitlab # [ 212.204905] gitaly[1397]: time="2026-07-09T06:52:21.008Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC3S1G90GP81MBJHGSP7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:50.105" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:21.006" grpc.time_ms=2.027 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab # [ 212.231312] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.032Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"56b6b01ed8fe7f6e2b7b38fe","created_at":"2026-07-09T06:52:19.448Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-85c8e16108aa40f2","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.454Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56b6b01ed8fe7f6e2b7b38fe: done: 0.858708 sec","job_status":"done","queue_duration_s":0.718926,"scheduling_latency_s":0.71895,"gitaly_calls":3,"gitaly_duration_s":0.051139,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.073505,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002751,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004807,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.015237,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.010742,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.004873,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000539,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.031494,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.454,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.056,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.1,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.196,"db_ci_duration_s":0.259,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.13924,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.737992398,"duration_s":0.858708,"completed_at":"2026-07-09T06:52:21.032Z","load_balancing_strategy":"primary","db_duration_s":0.578757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1975gitlab # [ 212.313316] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.040Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bb3f4cbc1fee517d06f37724"],"class":"WebHooks::LogExecutionWorker","jid":"ae5a73aa80952cab17e1e693","created_at":"2026-07-09T06:52:19.911Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-20a2085d91e84699","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2bb16ab6831798fa084469ad062fc4ad8ac966dcd5e66a03480ba1db33bc4720","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.919Z","job_size_bytes":2506,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ae5a73aa80952cab17e1e693: start","job_status":"start","queue_duration_s":1.120448,"scheduling_latency_s":1.120486}1976gitlab # [ 212.339383] gitaly[1397]: time="2026-07-09T06:52:21.037Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:31.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:21.028" grpc.time_ms=8.952 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab # [ 212.361497] nixbot[3109]: {"ts": "2026-07-09T06:52:21.135Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 \"HTTP/1.1 201 Created\""}1978gitlab # [ 212.363218] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 3161979gitlab # [ 212.368821] nixbot[3109]: {"ts": "2026-07-09T06:52:21.170Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""}1980gitlab # [ 212.369657] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4801981gitlab # [ 212.375068] nixbot[3109]: {"ts": "2026-07-09T06:52:21.178Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4628e241e0e4-1418f780", "argv0": "bwrap"}1982gitlab # [ 212.409782] gitaly[1397]: time="2026-07-09T06:52:21.213Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC68M9Q9BC6NY779HQ8EY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:51.110" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:21.211" grpc.time_ms=1.102 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 212.458478] nix-daemon[3445]: accepted connection from pid 3550, user nixbot1984gitlab # [ 212.463717] nix-daemon[3445]: accepted connection from pid 3552, user nixbot1985gitlab # [ 212.481260] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.283Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bb3f4cbc1fee517d06f37724"],"class":"WebHooks::LogExecutionWorker","jid":"ae5a73aa80952cab17e1e693","created_at":"2026-07-09T06:52:19.911Z","trace_propagation_headers":{"sentry-trace":"18c50f27c8494fac82ddc9d84f7e306e-20a2085d91e84699","baggage":"sentry-trace_id=18c50f27c8494fac82ddc9d84f7e306e,sentry-release=Unknown"},"correlation_id":"01KX2TC2SR77YMGK7M7MH2XEE9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2bb16ab6831798fa084469ad062fc4ad8ac966dcd5e66a03480ba1db33bc4720","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:19.919Z","job_size_bytes":2506,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ae5a73aa80952cab17e1e693: done: 0.24305 sec","job_status":"done","queue_duration_s":1.120448,"scheduling_latency_s":1.120486,"redis_calls":10,"redis_duration_s":0.019931,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.051,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.051,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00591388,"duration_s":0.24305,"completed_at":"2026-07-09T06:52:21.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1986gitlab # [ 212.555280] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8e5242ea24f29ded20f4c475","created_at":"2026-07-09T06:52:20.617Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-50011061ea3b40bd","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.624Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e5242ea24f29ded20f4c475: start","job_status":"start","queue_duration_s":0.73313,"scheduling_latency_s":0.733199}1987gitlab # [ 212.581247] gitaly[1397]: time="2026-07-09T06:52:21.382Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC68M9Q9BC6NY779HQ8EY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:31.209" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:21.371" grpc.time_ms=10.527 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab # [ 212.654156] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 3161989gitlab # [ 212.656482] nixbot[3109]: {"ts": "2026-07-09T06:52:21.459Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8cb378872b8fe1f7cde149c561f5802ee16ad182 \"HTTP/1.1 201 Created\""}1990gitlab # [ 212.691333] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.494Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8e5242ea24f29ded20f4c475","created_at":"2026-07-09T06:52:20.617Z","trace_propagation_headers":{"sentry-trace":"60189626cbf94157843cc30414959e69-50011061ea3b40bd","baggage":"sentry-trace_id=60189626cbf94157843cc30414959e69,sentry-release=Unknown"},"correlation_id":"01KX2TC3S1G90GP81MBJHGSP7G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.624Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e5242ea24f29ded20f4c475: done: 0.136607 sec","job_status":"done","queue_duration_s":0.73313,"scheduling_latency_s":0.733199,"redis_calls":10,"redis_duration_s":0.02997,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001966,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.026785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001347657,"duration_s":0.136607,"completed_at":"2026-07-09T06:52:21.494Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 212.779970] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.582Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"83de3b3545495bddbf4482f6","created_at":"2026-07-09T06:52:20.993Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.997Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-83de3b3545495bddbf4482f6: start","job_status":"start","queue_duration_s":0.584686,"scheduling_latency_s":0.584761}1992gitlab # [ 212.889243] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.691Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"83de3b3545495bddbf4482f6","created_at":"2026-07-09T06:52:20.993Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.997Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-83de3b3545495bddbf4482f6: done: 0.109372 sec","job_status":"done","queue_duration_s":0.584686,"scheduling_latency_s":0.584761,"redis_calls":7,"redis_duration_s":0.009635000000000001,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00168,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.109372,"completed_at":"2026-07-09T06:52:21.691Z","load_balancing_strategy":"primary","db_duration_s":0.010842,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1993gitlab # [ 212.976500] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.779Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b800a8f25e64d0e79d3118a8","created_at":"2026-07-09T06:52:20.998Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.999Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b800a8f25e64d0e79d3118a8: start","job_status":"start","queue_duration_s":0.778949,"scheduling_latency_s":0.779027}1994gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses1995gitlab # [ 213.158186] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:21.960Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b800a8f25e64d0e79d3118a8","created_at":"2026-07-09T06:52:20.998Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:20.999Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b800a8f25e64d0e79d3118a8: done: 0.181652 sec","job_status":"done","queue_duration_s":0.778949,"scheduling_latency_s":0.779027,"redis_calls":10,"redis_duration_s":0.024965,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.01244,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.181652,"completed_at":"2026-07-09T06:52:21.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996gitlab # [ 213.268804] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.071Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"54f80a7955d6d78610868d48","created_at":"2026-07-09T06:52:21.110Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-b06a4f9716da4413","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.115Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-54f80a7955d6d78610868d48: start","job_status":"start","queue_duration_s":0.956056,"scheduling_latency_s":0.956122}1997gitlab # [ 213.350777] gitaly[1397]: time="2026-07-09T06:52:22.153Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC75DT7FV2EDZZG7MP4RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:52.049" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.150" grpc.time_ms=2.586 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 213.375353] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1161999gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.17 seconds)2000gitlab # [ 213.479405] gitaly[1397]: time="2026-07-09T06:52:22.282Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:22.282" grpc.time_ms=0.481 pid=1397 span.kind=server system=grpc2001gitlab # [ 213.517474] gitaly[1397]: time="2026-07-09T06:52:22.320Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:22.320" grpc.time_ms=0.458 pid=1397 span.kind=server system=grpc2002gitlab # [ 213.556861] gitaly[1397]: time="2026-07-09T06:52:22.360Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:22.359" grpc.time_ms=0.499 pid=1397 span.kind=server system=grpc2003gitlab # [ 213.583923] gitaly[1397]: time="2026-07-09T06:52:22.387Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T diskcache=922b4e1f-2fb7-48be-95bc-5d6c1a95f090 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:52.249" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:22.249" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2004gitlab # [ 213.585237] gitaly[1397]: time="2026-07-09T06:52:22.387Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=95 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7046 command.oublock=0 command.real_time_ms=127 command.spawn_token_fork_ms=2 command.system_time_ms=53 command.user_time_ms=42 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:52.249" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:22.249" grpc.time_ms=137.774 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2005gitlab # [ 213.710426] gitaly[1397]: time="2026-07-09T06:52:22.513Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:52.512" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.512" grpc.time_ms=1.326 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2006gitlab # [ 213.800182] gitaly[1397]: time="2026-07-09T06:52:22.603Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:52.600" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.601" grpc.time_ms=1.192 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2007gitlab # [ 213.908582] gitaly[1397]: time="2026-07-09T06:52:22.711Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5YGKMY795GPT5TWSD3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:52.708" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:22.709" grpc.time_ms=1.58 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2008gitlab # [ 213.935770] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.738Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"54f80a7955d6d78610868d48","created_at":"2026-07-09T06:52:21.110Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-b06a4f9716da4413","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.115Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2147,"job_status":"rescheduled","message":"PipelineProcessWorker JID-54f80a7955d6d78610868d48: rescheduled"}2009gitlab # [ 213.969436] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.771Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"54f80a7955d6d78610868d48","created_at":"2026-07-09T06:52:21.110Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-b06a4f9716da4413","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.115Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-54f80a7955d6d78610868d48: done: 0.700455 sec","job_status":"done","queue_duration_s":0.956056,"scheduling_latency_s":0.956122,"gitaly_calls":4,"gitaly_duration_s":0.188707,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.10816,"redis_read_bytes":185,"redis_write_bytes":23557,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.003765,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.007413,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.02942,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.016325,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12389,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.030963,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00174,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002657,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012112,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.115,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.025,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.056,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.257353,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.58126382,"duration_s":0.700455,"completed_at":"2026-07-09T06:52:22.771Z","load_balancing_strategy":"primary","db_duration_s":0.125395,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2010gitlab # [ 214.046694] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:22.785Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"93d44d0f3a0b7baf47f8071e","created_at":"2026-07-09T06:52:21.150Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.153Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-93d44d0f3a0b7baf47f8071e: start","job_status":"start","queue_duration_s":1.631303,"scheduling_latency_s":1.631367}2011gitlab # [ 214.227209] gitaly[1397]: time="2026-07-09T06:52:23.030Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:53.027" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.028" grpc.time_ms=1.685 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012gitlab # [ 214.318196] gitaly[1397]: time="2026-07-09T06:52:23.121Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:53.118" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.119" grpc.time_ms=1.484 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2013gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses2014gitlab # [ 214.402257] gitaly[1397]: time="2026-07-09T06:52:23.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC5TKKVT2VAJAKT6N41HQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:53.201" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.202" grpc.time_ms=1.554 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015gitlab # [ 214.473119] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.274Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"93d44d0f3a0b7baf47f8071e","created_at":"2026-07-09T06:52:21.150Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-0b6dc85f71e146af","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:21.153Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-93d44d0f3a0b7baf47f8071e: done: 0.489425 sec","job_status":"done","queue_duration_s":1.631303,"scheduling_latency_s":1.631367,"gitaly_calls":3,"gitaly_duration_s":0.028879,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.08449000000000001,"redis_read_bytes":99,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.006994,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.013934,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.004739,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.014397,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001145,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00219,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.034097,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.09,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.055,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.251242,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.304014337,"duration_s":0.489425,"completed_at":"2026-07-09T06:52:23.274Z","load_balancing_strategy":"primary","db_duration_s":0.098825,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2016gitlab # [ 214.559421] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"696a190f7bfc5d3b5a95c5ff","created_at":"2026-07-09T06:52:22.224Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.232Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-696a190f7bfc5d3b5a95c5ff: start","job_status":"start","queue_duration_s":1.129919,"scheduling_latency_s":1.129981}2017gitlab # [ 214.577440] gitaly[1397]: time="2026-07-09T06:52:23.379Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC8AV87B5KZK1VDBZ09Q8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:53.276" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:23.377" grpc.time_ms=2.272 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018gitlab # [ 214.617334] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1602019gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.24 seconds)2020gitlab # [ 214.879191] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.681Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"696a190f7bfc5d3b5a95c5ff","created_at":"2026-07-09T06:52:22.224Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.232Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-696a190f7bfc5d3b5a95c5ff: done: 0.319939 sec","job_status":"done","queue_duration_s":1.129919,"scheduling_latency_s":1.129981,"redis_calls":15,"redis_duration_s":0.033391000000000004,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.002834,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.016042,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.004345,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005737,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140465,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002698108,"duration_s":0.319939,"completed_at":"2026-07-09T06:52:23.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056747,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2021gitlab # [ 214.963302] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:23.765Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"b539a2a4dc2247b662af9bf7","created_at":"2026-07-09T06:52:23.655Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-712dbd74b7e3446b","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d9e4da9715e14a7006fd5e12e0b6b729123be290a75bfd00943e204e64372d4b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:23.662Z","job_size_bytes":464,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-b539a2a4dc2247b662af9bf7: start","job_status":"start","queue_duration_s":0.103321,"scheduling_latency_s":0.103359}2022gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses2023gitlab # [ 215.145443] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:23.947Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"b539a2a4dc2247b662af9bf7","created_at":"2026-07-09T06:52:23.655Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-712dbd74b7e3446b","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:d9e4da9715e14a7006fd5e12e0b6b729123be290a75bfd00943e204e64372d4b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:23.662Z","job_size_bytes":464,"pid":2147,"sidekiq_tid":"m2xb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-b539a2a4dc2247b662af9bf7: fail: 0.180573 sec","job_status":"fail","queue_duration_s":0.103321,"scheduling_latency_s":0.103359,"redis_calls":7,"redis_duration_s":0.016118,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.180573,"completed_at":"2026-07-09T06:52:23.946Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.036814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 215.773373] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.575Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1ba10ec184a8a29b7fcac598","created_at":"2026-07-09T06:52:22.395Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.403Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1ba10ec184a8a29b7fcac598: start","job_status":"start","queue_duration_s":2.17162,"scheduling_latency_s":2.171657}2025gitlab # [ 215.823150] gitaly[1397]: time="2026-07-09T06:52:24.626Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TC9KNXCDFCM5FBP8YJXSF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:54.523" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:24.625" grpc.time_ms=0.729 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2026gitlab # [ 215.854160] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 912027gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.24 seconds)2028gitlab # [ 215.880765] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.683Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1ba10ec184a8a29b7fcac598","created_at":"2026-07-09T06:52:22.395Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.403Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1ba10ec184a8a29b7fcac598: done: 0.108496 sec","job_status":"done","queue_duration_s":2.17162,"scheduling_latency_s":2.171657,"redis_calls":10,"redis_duration_s":0.017474,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001301562,"duration_s":0.108496,"completed_at":"2026-07-09T06:52:24.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2029gitlab # [ 215.916965] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.688Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7d867453a43cfca4be16bf11","created_at":"2026-07-09T06:52:22.420Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.436Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7d867453a43cfca4be16bf11: start","job_status":"start","queue_duration_s":2.251899,"scheduling_latency_s":2.251931}2030gitlab # [ 216.035736] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.838Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7d867453a43cfca4be16bf11","created_at":"2026-07-09T06:52:22.420Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.436Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7d867453a43cfca4be16bf11: done: 0.149629 sec","job_status":"done","queue_duration_s":2.251899,"scheduling_latency_s":2.251931,"redis_calls":10,"redis_duration_s":0.019827,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002875,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.072,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003090337,"duration_s":0.149629,"completed_at":"2026-07-09T06:52:24.837Z","load_balancing_strategy":"primary","db_duration_s":0.074475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031gitlab # [ 216.110408] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"996009808a53ca940b216e2e","created_at":"2026-07-09T06:52:22.450Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.459Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-996009808a53ca940b216e2e: start","job_status":"start","queue_duration_s":2.390456,"scheduling_latency_s":2.390517}2032gitlab # [ 216.174828] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.977Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"996009808a53ca940b216e2e","created_at":"2026-07-09T06:52:22.450Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.459Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-996009808a53ca940b216e2e: done: 0.127656 sec","job_status":"done","queue_duration_s":2.390456,"scheduling_latency_s":2.390517,"redis_calls":13,"redis_duration_s":0.040022,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004233,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.020422,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003181689,"duration_s":0.127656,"completed_at":"2026-07-09T06:52:24.977Z","load_balancing_strategy":"primary","db_duration_s":0.01617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033gitlab # [ 216.211759] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:24.988Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dff925da3166ffc0bf985349","created_at":"2026-07-09T06:52:22.460Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.468Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dff925da3166ffc0bf985349: start","job_status":"start","queue_duration_s":2.519201,"scheduling_latency_s":2.519271}2034gitlab # [ 216.353769] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.156Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"dff925da3166ffc0bf985349","created_at":"2026-07-09T06:52:22.460Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.468Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-dff925da3166ffc0bf985349: done: 0.168359 sec","job_status":"done","queue_duration_s":2.519201,"scheduling_latency_s":2.519271,"redis_calls":10,"redis_duration_s":0.084706,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004333,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014567,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.065806,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003178058,"duration_s":0.168359,"completed_at":"2026-07-09T06:52:25.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035gitlab # [ 216.456640] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.259Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ab9fad311132c5cba097f5e5","created_at":"2026-07-09T06:52:22.469Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.479Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ab9fad311132c5cba097f5e5: start","job_status":"start","queue_duration_s":2.780167,"scheduling_latency_s":2.78024}2036gitlab # [ 216.559773] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ab9fad311132c5cba097f5e5","created_at":"2026-07-09T06:52:22.469Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.479Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ab9fad311132c5cba097f5e5: done: 0.103062 sec","job_status":"done","queue_duration_s":2.780167,"scheduling_latency_s":2.78024,"redis_calls":11,"redis_duration_s":0.024413,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003953,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.016613,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003847,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001464991,"duration_s":0.103062,"completed_at":"2026-07-09T06:52:25.362Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2037gitlab # [ 216.659421] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.461Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c72e6dbbeb1b5ba4e689c873","created_at":"2026-07-09T06:52:22.482Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.492Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c72e6dbbeb1b5ba4e689c873: start","job_status":"start","queue_duration_s":2.969429,"scheduling_latency_s":2.969503}2038gitlab # [ 216.756725] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.559Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c72e6dbbeb1b5ba4e689c873","created_at":"2026-07-09T06:52:22.482Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.492Z","job_size_bytes":65,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c72e6dbbeb1b5ba4e689c873: done: 0.097646 sec","job_status":"done","queue_duration_s":2.969429,"scheduling_latency_s":2.969503,"redis_calls":10,"redis_duration_s":0.021482,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004112,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003429486,"duration_s":0.097646,"completed_at":"2026-07-09T06:52:25.559Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses2040gitlab # [ 216.855407] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.657Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c9a3ccff2cc3416214eef96c","created_at":"2026-07-09T06:52:22.496Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.501Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c9a3ccff2cc3416214eef96c: start","job_status":"start","queue_duration_s":3.155874,"scheduling_latency_s":3.15595}2041gitlab # [ 216.959263] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.762Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c9a3ccff2cc3416214eef96c","created_at":"2026-07-09T06:52:22.496Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.501Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c9a3ccff2cc3416214eef96c: done: 0.105471 sec","job_status":"done","queue_duration_s":3.155874,"scheduling_latency_s":3.15595,"redis_calls":7,"redis_duration_s":0.013316,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105471,"completed_at":"2026-07-09T06:52:25.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2042gitlab # [ 216.995692] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.772Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"180b8170d620fdc924a1aae5","created_at":"2026-07-09T06:52:22.607Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.627Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-180b8170d620fdc924a1aae5: start","job_status":"start","queue_duration_s":3.145014,"scheduling_latency_s":3.145075}2043gitlab # [ 216.997382] gitaly[1397]: time="2026-07-09T06:52:25.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=13 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=13 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCAQQQMPTAGVTE38FM2VC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:55.682" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:25.783" grpc.time_ms=14.821 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044gitlab # [ 217.064706] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1012045gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.21 seconds)2046gitlab # [ 217.142597] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:25.945Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"180b8170d620fdc924a1aae5","created_at":"2026-07-09T06:52:22.607Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.627Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-180b8170d620fdc924a1aae5: done: 0.172994 sec","job_status":"done","queue_duration_s":3.145014,"scheduling_latency_s":3.145075,"redis_calls":10,"redis_duration_s":0.01807,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002882,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002856787,"duration_s":0.172994,"completed_at":"2026-07-09T06:52:25.945Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 217.228139] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.030Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a5bd3c1a0ba23b6d5b0c64ef","created_at":"2026-07-09T06:52:22.631Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.646Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a5bd3c1a0ba23b6d5b0c64ef: start","job_status":"start","queue_duration_s":3.384512,"scheduling_latency_s":3.384546}2048gitlab # [ 217.307439] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.110Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a5bd3c1a0ba23b6d5b0c64ef","created_at":"2026-07-09T06:52:22.631Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.646Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a5bd3c1a0ba23b6d5b0c64ef: done: 0.079805 sec","job_status":"done","queue_duration_s":3.384512,"scheduling_latency_s":3.384546,"redis_calls":10,"redis_duration_s":0.024026000000000002,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001902,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.020039,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000774959,"duration_s":0.079805,"completed_at":"2026-07-09T06:52:26.110Z","load_balancing_strategy":"primary","db_duration_s":0.005296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049gitlab # [ 217.375542] nix-daemon[3445]: accepted connection from pid 3617, user nixbot2050gitlab # [ 217.377351] gitaly[1397]: time="2026-07-09T06:52:26.178Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.140" grpc.time_ms=0.717 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051gitlab # [ 217.392416] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.195Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8d73aa45c91e22206fc654be","created_at":"2026-07-09T06:52:22.742Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.750Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8d73aa45c91e22206fc654be: start","job_status":"start","queue_duration_s":3.445271,"scheduling_latency_s":3.445324}2052gitlab # [ 217.413791] gitaly[1397]: time="2026-07-09T06:52:26.217Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:36.184" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.205" grpc.time_ms=11.66 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 217.467295] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 1722054gitlab # [ 217.468471] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.270Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8d73aa45c91e22206fc654be","created_at":"2026-07-09T06:52:22.742Z","trace_propagation_headers":{"sentry-trace":"5a69f81cd83f47da8d397f51c070054f-f3fba3b6074543a8","baggage":"sentry-trace_id=5a69f81cd83f47da8d397f51c070054f,sentry-release=Unknown"},"correlation_id":"01KX2TC5YGKMY795GPT5TWSD3T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.750Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8d73aa45c91e22206fc654be: done: 0.075278 sec","job_status":"done","queue_duration_s":3.445271,"scheduling_latency_s":3.445324,"redis_calls":8,"redis_duration_s":0.009764,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006187,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003577,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075278,"completed_at":"2026-07-09T06:52:26.270Z","load_balancing_strategy":"primary","db_duration_s":0.026306,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2055gitlab # [ 217.474922] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e7c61d1723f7a7f6772ac5be","created_at":"2026-07-09T06:52:22.989Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-c9abab287cf744ab","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.996Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e7c61d1723f7a7f6772ac5be: start","job_status":"start","queue_duration_s":3.282153,"scheduling_latency_s":3.28218}2056gitlab # [ 217.540344] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e7c61d1723f7a7f6772ac5be","created_at":"2026-07-09T06:52:22.989Z","trace_propagation_headers":{"sentry-trace":"8e009a77a05242309b1ff6f53c4274fb-c9abab287cf744ab","baggage":"sentry-trace_id=8e009a77a05242309b1ff6f53c4274fb,sentry-release=Unknown"},"correlation_id":"01KX2TC5TKKVT2VAJAKT6N41HQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:22.996Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e7c61d1723f7a7f6772ac5be: done: 0.065134 sec","job_status":"done","queue_duration_s":3.282153,"scheduling_latency_s":3.28218,"redis_calls":7,"redis_duration_s":0.006257,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001716,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065134,"completed_at":"2026-07-09T06:52:26.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057gitlab # [ 217.613346] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.345Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"72ea4ae898810bd7fbb3b43c","created_at":"2026-07-09T06:52:26.250Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-24ccff0da2fe4245","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.255Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-72ea4ae898810bd7fbb3b43c: start","job_status":"start","queue_duration_s":0.090428,"scheduling_latency_s":0.090456}2058gitlab # [ 217.641143] gitaly[1397]: time="2026-07-09T06:52:26.444Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:26.442" grpc.time_ms=1.96 pid=1397 span.kind=server system=grpc2059gitlab # [ 217.664578] gitaly[1397]: time="2026-07-09T06:52:26.468Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:26.467" grpc.time_ms=0.352 pid=1397 span.kind=server system=grpc2060gitlab # [ 217.666518] nixbot[3109]: {"ts": "2026-07-09T06:52:26.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""}2061gitlab # [ 217.691617] gitaly[1397]: time="2026-07-09T06:52:26.495Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-09T06:52:26.494" grpc.time_ms=0.311 pid=1397 span.kind=server system=grpc2062gitlab # [ 217.713477] gitaly[1397]: time="2026-07-09T06:52:26.516Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC diskcache=06c49cb2-1076-4062-9ced-8003f0f2dc41 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:56.417" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.418" pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063gitlab # [ 217.714977] gitaly[1397]: time="2026-07-09T06:52:26.517Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=83 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6967 command.oublock=0 command.real_time_ms=80 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-09T06:52:56.417" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.418" grpc.time_ms=98.62 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064gitlab # [ 217.718890] gitaly[1397]: time="2026-07-09T06:52:26.521Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCBF9J846FQDY1V70FB39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.419" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.520" grpc.time_ms=0.617 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065gitlab # [ 217.762101] gitaly[1397]: time="2026-07-09T06:52:26.565Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:56.564" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.564" grpc.time_ms=1.056 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2066gitlab # [ 217.784644] gitaly[1397]: time="2026-07-09T06:52:26.588Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.587" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.587" grpc.time_ms=0.628 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 217.813728] gitaly[1397]: time="2026-07-09T06:52:26.616Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCBF9J846FQDY1V70FB39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:36.138" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:26.608" grpc.time_ms=7.746 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068gitlab # [ 217.854092] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1832069gitlab # [ 217.855994] nixbot[3109]: {"ts": "2026-07-09T06:52:26.659Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""}2070gitlab # [ 218.018774] gitaly[1397]: time="2026-07-09T06:52:26.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:56.819" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:26.819" grpc.time_ms=1.071 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses2072gitlab # [ 218.030588] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.832Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"24bbd51d3a97cfa19b8aec54","created_at":"2026-07-09T06:52:26.830Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"72ea4ae898810bd7fbb3b43c","job_size_bytes":3,"pid":2147,"job_status":"deduplicated","message":"PipelineProcessWorker JID-24bbd51d3a97cfa19b8aec54: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2073gitlab # [ 218.034346] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.835Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"72ea4ae898810bd7fbb3b43c","created_at":"2026-07-09T06:52:26.250Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-24ccff0da2fe4245","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.255Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2147,"job_status":"rescheduled","message":"PipelineProcessWorker JID-72ea4ae898810bd7fbb3b43c: rescheduled"}2074gitlab # [ 218.039415] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.842Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"72ea4ae898810bd7fbb3b43c","created_at":"2026-07-09T06:52:26.250Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-24ccff0da2fe4245","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.255Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-72ea4ae898810bd7fbb3b43c: done: 0.497032 sec","job_status":"done","queue_duration_s":0.090428,"scheduling_latency_s":0.090456,"gitaly_calls":4,"gitaly_duration_s":0.114991,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.025389,"redis_read_bytes":262,"redis_write_bytes":23388,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001119,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00325,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000945,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.004948,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.009214,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000252,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001911,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002631,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.214,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.199,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144943,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.455183829,"duration_s":0.497032,"completed_at":"2026-07-09T06:52:26.842Z","load_balancing_strategy":"primary","db_duration_s":0.217451,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2075gitlab # [ 218.143687] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5ecbfb30e120439b199cb644","created_at":"2026-07-09T06:52:26.410Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.412Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5ecbfb30e120439b199cb644: start","job_status":"start","queue_duration_s":0.431702,"scheduling_latency_s":0.431728}2076gitlab # [ 218.213836] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5ecbfb30e120439b199cb644","created_at":"2026-07-09T06:52:26.410Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.412Z","job_size_bytes":23,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5ecbfb30e120439b199cb644: done: 0.039169 sec","job_status":"done","queue_duration_s":0.431702,"scheduling_latency_s":0.431728,"redis_calls":7,"redis_duration_s":0.003189,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039169,"completed_at":"2026-07-09T06:52:26.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00941,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2077gitlab # [ 218.222124] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.885Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4d0470cc38fd7f61f7041a00","created_at":"2026-07-09T06:52:26.526Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.529Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4d0470cc38fd7f61f7041a00: start","job_status":"start","queue_duration_s":0.356523,"scheduling_latency_s":0.356548}2078gitlab # [ 218.296056] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.910Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4d0470cc38fd7f61f7041a00","created_at":"2026-07-09T06:52:26.526Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.529Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4d0470cc38fd7f61f7041a00: done: 0.024286 sec","job_status":"done","queue_duration_s":0.356523,"scheduling_latency_s":0.356548,"redis_calls":7,"redis_duration_s":0.002929,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000822,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024286,"completed_at":"2026-07-09T06:52:26.909Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2079gitlab # [ 218.365276] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:26.912Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3a4f1c35693f2aeb3209be8d","created_at":"2026-07-09T06:52:26.537Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.539Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3a4f1c35693f2aeb3209be8d: start","job_status":"start","queue_duration_s":0.372604,"scheduling_latency_s":0.372628}2080gitlab # [ 218.366877] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.017Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3a4f1c35693f2aeb3209be8d","created_at":"2026-07-09T06:52:26.537Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.539Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3a4f1c35693f2aeb3209be8d: done: 0.105661 sec","job_status":"done","queue_duration_s":0.372604,"scheduling_latency_s":0.372628,"redis_calls":7,"redis_duration_s":0.00325,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.073,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105661,"completed_at":"2026-07-09T06:52:27.017Z","load_balancing_strategy":"primary","db_duration_s":0.073006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 218.434418] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.021Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"b090520daf566100bd301f34","created_at":"2026-07-09T06:52:26.549Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.550Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b090520daf566100bd301f34: start","job_status":"start","queue_duration_s":0.470983,"scheduling_latency_s":0.471032}2082gitlab # [ 218.435887] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.051Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"b090520daf566100bd301f34","created_at":"2026-07-09T06:52:26.549Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.550Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b090520daf566100bd301f34: done: 0.029552 sec","job_status":"done","queue_duration_s":0.470983,"scheduling_latency_s":0.471032,"redis_calls":10,"redis_duration_s":0.002908,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000768,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000731,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029552,"completed_at":"2026-07-09T06:52:27.051Z","load_balancing_strategy":"primary","db_duration_s":0.002964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083gitlab # [ 218.512530] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e487af4b9a28bf2467e574be","created_at":"2026-07-09T06:52:26.551Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.552Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e487af4b9a28bf2467e574be: start","job_status":"start","queue_duration_s":0.501017,"scheduling_latency_s":0.50106}2084gitlab # [ 218.519445] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e487af4b9a28bf2467e574be","created_at":"2026-07-09T06:52:26.551Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.552Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e487af4b9a28bf2467e574be: done: 0.023218 sec","job_status":"done","queue_duration_s":0.501017,"scheduling_latency_s":0.50106,"redis_calls":7,"redis_duration_s":0.0017759999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001321,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023218,"completed_at":"2026-07-09T06:52:27.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 218.593398] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.078Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"53f31fb29d42ea55d96ce935","created_at":"2026-07-09T06:52:26.552Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.554Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-53f31fb29d42ea55d96ce935: start","job_status":"start","queue_duration_s":0.523136,"scheduling_latency_s":0.52316}2086gitlab # [ 218.612316] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.098Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"53f31fb29d42ea55d96ce935","created_at":"2026-07-09T06:52:26.552Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.554Z","job_size_bytes":19,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-53f31fb29d42ea55d96ce935: done: 0.020881 sec","job_status":"done","queue_duration_s":0.523136,"scheduling_latency_s":0.52316,"redis_calls":8,"redis_duration_s":0.001892,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020881,"completed_at":"2026-07-09T06:52:27.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 218.682652] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.100Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e8404433e4d5887ca8e8fd93","created_at":"2026-07-09T06:52:26.555Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.556Z","job_size_bytes":66,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e8404433e4d5887ca8e8fd93: start","job_status":"start","queue_duration_s":0.543354,"scheduling_latency_s":0.543378}2088gitlab # [ 218.706269] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.116Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e8404433e4d5887ca8e8fd93","created_at":"2026-07-09T06:52:26.555Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.556Z","job_size_bytes":66,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e8404433e4d5887ca8e8fd93: done: 0.016227 sec","job_status":"done","queue_duration_s":0.543354,"scheduling_latency_s":0.543378,"redis_calls":7,"redis_duration_s":0.001702,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016227,"completed_at":"2026-07-09T06:52:27.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2089gitlab # [ 218.786548] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"53e3b805aae109cb61fbd1d4","created_at":"2026-07-09T06:52:26.557Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.559Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-53e3b805aae109cb61fbd1d4: start","job_status":"start","queue_duration_s":0.558783,"scheduling_latency_s":0.558804}2090gitlab # [ 218.799196] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.146Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"53e3b805aae109cb61fbd1d4","created_at":"2026-07-09T06:52:26.557Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.559Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-53e3b805aae109cb61fbd1d4: done: 0.028402 sec","job_status":"done","queue_duration_s":0.558783,"scheduling_latency_s":0.558804,"redis_calls":7,"redis_duration_s":0.0018549999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028402,"completed_at":"2026-07-09T06:52:27.146Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003886,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab # [ 218.907935] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.148Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"ded8e024f076b2166788b6c6","created_at":"2026-07-09T06:52:26.595Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.597Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ded8e024f076b2166788b6c6: start","job_status":"start","queue_duration_s":0.550593,"scheduling_latency_s":0.550631}2092gitlab # [ 218.909679] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.170Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"ded8e024f076b2166788b6c6","created_at":"2026-07-09T06:52:26.595Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.597Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ded8e024f076b2166788b6c6: done: 0.022224 sec","job_status":"done","queue_duration_s":0.550593,"scheduling_latency_s":0.550631,"redis_calls":7,"redis_duration_s":0.00179,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022224,"completed_at":"2026-07-09T06:52:27.170Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093gitlab # [ 219.011297] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.171Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9f3a73a5de0ed344d06a1eb8","created_at":"2026-07-09T06:52:26.598Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.599Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9f3a73a5de0ed344d06a1eb8: start","job_status":"start","queue_duration_s":0.572469,"scheduling_latency_s":0.572493}2094gitlab # [ 219.012981] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9f3a73a5de0ed344d06a1eb8","created_at":"2026-07-09T06:52:26.598Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.599Z","job_size_bytes":6,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9f3a73a5de0ed344d06a1eb8: done: 0.013708 sec","job_status":"done","queue_duration_s":0.572469,"scheduling_latency_s":0.572493,"redis_calls":7,"redis_duration_s":0.001596,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013708,"completed_at":"2026-07-09T06:52:27.185Z","load_balancing_strategy":"primary","db_duration_s":0.000426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095gitlab # [ 219.112660] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.186Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"84b9333438f37c11b04887a0","created_at":"2026-07-09T06:52:26.599Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.603Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-84b9333438f37c11b04887a0: start","job_status":"start","queue_duration_s":0.582969,"scheduling_latency_s":0.582991}2096gitlab # [ 219.114504] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"84b9333438f37c11b04887a0","created_at":"2026-07-09T06:52:26.599Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.603Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-84b9333438f37c11b04887a0: done: 0.029812 sec","job_status":"done","queue_duration_s":0.582969,"scheduling_latency_s":0.582991,"redis_calls":10,"redis_duration_s":0.002517,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000621,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391391,"duration_s":0.029812,"completed_at":"2026-07-09T06:52:27.216Z","load_balancing_strategy":"primary","db_duration_s":0.004173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2097gitlab # [ 219.142601] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.218Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"75801ddd344521dcf0c3828e","created_at":"2026-07-09T06:52:26.835Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.836Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-75801ddd344521dcf0c3828e: start","job_status":"start","queue_duration_s":0.381147,"scheduling_latency_s":0.381167}2098gitlab # [ 219.216507] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.359Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"75801ddd344521dcf0c3828e","created_at":"2026-07-09T06:52:26.835Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-2af715b963f84138","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:26.836Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-75801ddd344521dcf0c3828e: done: 0.141103 sec","job_status":"done","queue_duration_s":0.381147,"scheduling_latency_s":0.381167,"gitaly_calls":3,"gitaly_duration_s":0.010853,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011912,"redis_read_bytes":98,"redis_write_bytes":6011,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001692,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003198,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002494,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000255,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00089,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091598,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.116770427,"duration_s":0.141103,"completed_at":"2026-07-09T06:52:27.359Z","load_balancing_strategy":"primary","db_duration_s":0.03544,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2099gitlab # [ 219.248297] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9a190ff218e5aef8eaf4677a","created_at":"2026-07-09T06:52:27.272Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-011c036e35534b1a","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:27.274Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9a190ff218e5aef8eaf4677a: start","job_status":"start","queue_duration_s":0.086835,"scheduling_latency_s":0.086868}2100gitlab # [ 219.316803] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:27.402Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9a190ff218e5aef8eaf4677a","created_at":"2026-07-09T06:52:27.272Z","trace_propagation_headers":{"sentry-trace":"fd7fcd51bb8f4b16a8d7f71c9149571c-011c036e35534b1a","baggage":"sentry-trace_id=fd7fcd51bb8f4b16a8d7f71c9149571c,sentry-release=Unknown"},"correlation_id":"01KX2TCB3HRMD93BD0GRGGSCSC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:27.274Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9a190ff218e5aef8eaf4677a: done: 0.041261 sec","job_status":"done","queue_duration_s":0.086835,"scheduling_latency_s":0.086868,"redis_calls":7,"redis_duration_s":0.002725,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041261,"completed_at":"2026-07-09T06:52:27.402Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2101gitlab # [ 219.340792] gitaly[1397]: time="2026-07-09T06:52:27.286Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-09T06:52:57.284" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:27.285" grpc.time_ms=1.144 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102gitlab # [ 219.419229] gitaly[1397]: time="2026-07-09T06:52:27.315Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:57.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:27.314" grpc.time_ms=0.578 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2103gitlab # [ 219.420598] gitaly[1397]: time="2026-07-09T06:52:27.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCB3HRMD93BD0GRGGSCSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:57.342" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:27.343" grpc.time_ms=1.049 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab # [ 219.532801] gitaly[1397]: time="2026-07-09T06:52:28.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCD86EQ7S95092DK22R62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:58.233" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:28.334" grpc.time_ms=1.244 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2105gitlab # [ 219.579114] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 852106gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 1.53 seconds)2107gitlab # [ 219.668997] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:28.471Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:28.467Z","retry_count":0,"job_size_bytes":451,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: start","job_status":"start","queue_duration_s":0.003517,"scheduling_latency_s":0.003588}2108gitlab # [ 219.762197] sidekiq-cluster[2147]: {"severity":"WARN","time":"2026-07-09T06:52:28.564Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"2e28f00dfbd44ec845d9f76f","created_at":"2026-07-09T06:52:03.279Z","trace_propagation_headers":{"sentry-trace":"061fb86fd6484e9ba28c8e8040eccec1-dcd0ae1de21e48f2","baggage":"sentry-trace_id=061fb86fd6484e9ba28c8e8040eccec1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TBMDPE961CPQ6NT2P4CPQ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:f204d91a34b669040ce6ef9ba5775f36f95b247b33712ac224f36df15364d890","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:28.467Z","retry_count":0,"job_size_bytes":451,"pid":2147,"sidekiq_tid":"ma2v","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-2e28f00dfbd44ec845d9f76f: fail: 0.093161 sec","job_status":"fail","queue_duration_s":0.003517,"scheduling_latency_s":0.003588,"redis_calls":7,"redis_duration_s":0.0032170000000000002,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093161,"completed_at":"2026-07-09T06:52:28.564Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.006857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses2110gitlab # [ 220.730540] gitaly[1397]: time="2026-07-09T06:52:29.532Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCECFBRD9WH63TCBHRM25 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:52:59.430" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:29.530" grpc.time_ms=1.422 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2111gitlab # [ 220.797668] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1432112gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.21 seconds)2113gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses2114gitlab # [ 221.813803] nix-daemon[3445]: accepted connection from pid 3685, user nixbot2115gitlab # [ 221.898437] gitaly[1397]: time="2026-07-09T06:52:30.700Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCFHVYV47RECF3NMMSAG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:53:00.598" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:30.699" grpc.time_ms=0.712 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 221.936334] gitaly[1397]: time="2026-07-09T06:52:30.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KX2TCFHVYV47RECF3NMMSAG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-09T06:52:40.257" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-09T06:52:30.727" grpc.time_ms=11.7 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab # [ 221.979485] sidekiq-cluster[2147]: {"severity":"INFO","time":"2026-07-09T06:52:30.782Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"54810a033250b68d627c0fbc","created_at":"2026-07-09T06:52:30.777Z","trace_propagation_headers":{"sentry-trace":"a282f290df694797b2b64649dcf2747d-8a915df8d107472b","baggage":"sentry-trace_id=a282f290df694797b2b64649dcf2747d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KX2TCFHVYV47RECF3NMMSAG5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-09T06:52:30.780Z","job_size_bytes":3,"pid":2147,"sidekiq_tid":"m7sj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-54810a033250b68d627c0fbc: start","job_status":"start","queue_duration_s":0.001516,"scheduling_latency_s":0.001553}2118gitlab # [ 221.994315] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1452119gitlab # [ 221.997553] gitaly[1397]: time="2026-07-09T06:52:30.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KX2TCFJ35AQ1Y357RJ8GFVTS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-09T06:53:00.699" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-09T06:52:30.799" grpc.time_ms=0.903 pid=1397 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2120gitlab # [ 222.020704] gitlab-workhorse[1306]: gitlab 2001:db8:1::1 - - [2026/07/09:06:52:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 1642121gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf/statuses, in 0.22 seconds)2122(finished: subtest: eval and build post commit statuses, in 10.14 seconds)2123(finished: run the VM test script, in 258.02 seconds)2124gitlab # [ 222.032603] nixbot[3109]: {"ts": "2026-07-09T06:52:30.835Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf \"HTTP/1.1 201 Created\""}2125test script finished in 258.06s2126cleanup2127kill QemuMachine (pid 13)2128gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2129(finished: cleanup, in 0.02 seconds)2130additionally exposed symbols:2131 gitlab,2132 vlan1,2133 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2134[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-09T06:52:16.803Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2135[]2136[]2137[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-09T06:52:20.675959Z', 'started_at': None, 'finished_at': None}]2138[]2139[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2140[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2141[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2142[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2143[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:26.242Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-09T06:52:26.638Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2144[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:26.242Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-09T06:52:26.638Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2145[{'id': 2, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-09T06:52:21.140Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:26.242Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '4628e241e0e4d84f4ce0118b81ba70d1f3de2fdf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-09T06:52:26.638Z', 'started_at': None, 'finished_at': '2026-07-09T06:52:30.768Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]